#420 Add experimental support for 'FB' format

[yruslan]

Resolves #420
Resolves #422

Added experimental support for fixed block record formats (FB). When the record format is 'FB' you need to specify either block length or number of records per
block. As with 'F' if record_length is not specified, it will be determined from the copybook.

.option("record_format", "FB")
.option("record_length", "250")
.option("block_length", "500")

or

.option("record_format", "FB")
.option("record_length", "250")
.option("records_per_block", "2")

More on fixed-length record formats: https://www.ibm.com/docs/en/zos/2.3.0?topic=sets-fixed-length-record-formats

[sree018]

Hi @yruslan

I have four types of files in ebcdic format
1. variable block with variable length record (VB-> able to parse file )
2. variable block with fixed length record ( not able to parse )
Custom record extractor implementation for VBVR data files, should it not use rawRecordExtractor?#412
I am using above logic to convert file to fixed block and reading as F.
3. fixed block with variable length record (V-> able to parse file)
4. fixed block with fixed length record (F-> able to parse file )

File_Type   BDW.   RDW  
 VBVR         yes      yes 
 VBFR          yes      no
 FBVR          no       yes
 FBFR          no         no 

if you provide record_format as VBVR,VBFR,FBVR and FBFR which will help us.
Unfortunately, due to security issues, I cant provide dummy files for variable block with fixed length files.

[yruslan]

@sree018 , I see that you can't provide a dummy data. But could you please provide a spec for VBFR?

The way I see it:
VBVR: BDW RDW DATA RDW DATA RDW DATA BDW RDW DATA RDW DATA RDW DATA (aka VB)
VBFR: ? (has BDW, but not RDW??? I couldn't find a spec for such kind of files in IBM'd docs)
FBVR: RDW DATA RDW DATA RDW DATA RDW DATA RDW DATA RDW DATA (aka V, no BDW)
FBFR: DATA DATA DATA DATA DATA DATA (aka F and FB)

Currently Cobrix supports this directly:
VBVR : option("record_format", "VB")
VBFR : ?
FBVR : option("record_format", "V")
FBFR : option("record_format", "F")

Alternatively, if you can't provide dummy data or a spec, you can implement VBFR as a custom record extractor and submit a PR. We will integrate it with Cobrix and add a direct suport.

[yruslan]

Does this make any sense?
VBFR: BDW DATA DATA DATA BDW DATA DATA DATA

I can add support for BDWs for .option("record_format", "FB") and this might solve the problem.

[sree018]

@yruslan thanks for info.
My file follow this pattern.
VBFR: BDW DATA DATA DATA BDW DATA DATA DATA
DATA is fixed length.
But, In my copy book we flattened RDW.
after adjusting copy book, I am able parse VBFR-> BDW + RDW(2,2)

[yruslan]

I'm glad you found a solution. I'll add support for
VBFR: BDW DATA DATA DATA BDW DATA DATA DATA
as well, for completion.

[yruslan]

@sree018, just added support for BDW without RDW. Use

.option("record_format", "FB")
.option("record_length", "250")

(250 - record length)
or

.option("record_format", "FB")

[sree018]

Hi @yruslan

Incase of multi segment hierarchical copybook,

by using custom record extractor, how do i parse file ?

[sree018]

Hi @yruslan

I have multi segmented binary file and able to parse it, if single segment column present. How do I parse multi segmented file with two segment columns present ?

ex: copybook

       01 MASTER-DATA.
          05 KEY-TYPE  PIC X(01).
          05 KEY-IND     PIC X(01).
 * KEY=A AND IND=0
          05 KEY-A0-REC.
                 10 DATA-FIELDS  PIC X(998).
   *KEY =B AND IND=0
          05 KEY-B0-REC REDEFINES KEY-A0-REC.
                 10 DATA-FIELDS PIC X(998).
  *KEY= 'A'-'F' AND IND=1
           05 KEY-01-REC REDEFINES KEY-B0-REC.
                 10 DATA-FIELDS PIC X(998).
  * KEY = 'A-F' AND IND=2
          05 KEY-02-REC REDEFINES KEY-01-REC.
                 10 DATA-FIELDS PIC X(998).

[yruslan]

Hi @sree018,

Please, create a new issue for this question.