Build only what is needed

[JVoogt]

Hi, For performance reasons I needed to only build the objects that I really needed. Our FixedWidth files contains 14,000+ columns and normally I only require around 50 of these columns. I built the object .json file for the entire 14,000+ but then filter the object down to only what I need. Has to comment out the validation part.

` headers = ['HEAD001', 'HEAD002', 'HEAD005'] configfile = 'layout.json'

with open(configfile, 'r') as f:
    config = json.load(f)

#Create FixedWidth Object
fw_config = deepcopy(config)
fw_configFiltered = {x: k for x, k in fw_config.items() if x in (headers)}`

Please see below part that I had to comment out, was thinking of rather adding a parameter to ignore if I need this to be ignored. Is there perhaps a better way of doing this?

` #ensure start_pos and end_pos or length is correct in config # current_pos = 1 # for start_pos, field_name in self.ordered_fields:

    #     if start_pos != current_pos:
    #         raise ValueError("Field %s starts at position %d; \
    #         should be %d (or previous field definition is incorrect)." \
    #         % (field_name, start_pos, current_pos))

    #     current_pos = current_pos + config[field_name]['length']`

[ShawnMilo]

I don't think I understand your requirements. Do you only need a subset of the fields when reading or writing? Do you never need the others? Seems like you could just write the config for the 50 fields and account for the empty space by just creating a bunch of non-required text fields that default to space padding.