hop icon indicating copy to clipboard operation
hop copied to clipboard

[Bug]: ETL metadata injection CSV issue

Open enjoyy2 opened this issue 7 months ago • 1 comments

Apache Hop version?

2.13

Java version?

24

Operating system

Windows

What happened?

testing the samples:

process-files-with-mdi.hpl process-files-with-mdi-template.hpl

it is supposed to output a unified file.

the file is created but, the fields do not have the enclosure "", although on the etl metadata injection on the text file output it is set correctly to ", even if i change it to -- it does not work. the delimiter works correctly when changed to something else.

first_name,last_name,primary_email,address_zip,address_country Ralph,Padilla,[email protected],35528,Chile Colton,Cobb,[email protected],9351-5888,Austria Orlando,Sheppard,[email protected],134388,Russian Federation Fiona,Fleming,[email protected],28748,Austria Tatum,Chen,[email protected],28555,Austria Elton,Barry,[email protected],54760,South Africa Gray,Hodge,[email protected],54655,Netherlands Zenaida,Hicks,[email protected],423468,Philippines Pamela,Vargas,[email protected],6011,Nigeria Rhoda,Jennings,[email protected],5424,Costa Rica Ralph,Padilla,[email protected],35528,Chile Colton,Cobb,[email protected],9351-5888,Austria Orlando,Sheppard,[email protected],134388,Russian Federation Fiona,Fleming,[email protected],28748,Austria Tatum,Chen,[email protected],28555,Austria Ralph,Padilla,[email protected],35528,Chile Colton,Cobb,[email protected],9351-5888,Austria Orlando,Sheppard,[email protected],134388,Russian Federation Fiona,Fleming,[email protected],28748,Austria Tatum,Chen,[email protected],28555,Austria

new delimiter Ralph;Padilla;[email protected];35528;Chile Colton;Cobb;[email protected];9351-5888;Austria Orlando;Sheppard;[email protected];134388;Russian Federation Fiona;Fleming;[email protected];28748;Austria Tatum;Chen;[email protected];28555;Austria

new enclosure set to -- Ralph;Padilla;[email protected];35528;Chile Colton;Cobb;[email protected];9351-5888;Austria Orlando;Sheppard;[email protected];134388;Russian Federation Fiona;Fleming;[email protected];28748;Austria Tatum;Chen;[email protected];28555;Austria

Issue Priority

Priority: 1

Issue Component

Component: Actions, Component: Transforms

enjoyy2 avatar Jun 26 '25 13:06 enjoyy2

also when mapping length the same way we map the field name or the field type in the text file output results in adding a right tab pad. even if the right pad is set to N

Ralph ;Padilla ;[email protected];35528 ;Chile Colton ;Cobb ;[email protected] ;9351-5888;Austria Orlando;Sheppard;[email protected] ;134388 ;Russian Federation Fiona ;Fleming ;[email protected] ;28748 ;Austria Tatum ;Chen ;[email protected] ;28555 ;Austria

enjoyy2 avatar Jun 26 '25 14:06 enjoyy2

Update (20 Sep 2025)

For anyone landing here later, I just re-tested this on Hop 2.15. Field enclosure works fine, Hop’s Text file output only adds the enclosure (e.g. " ) when a field needs it, typically when the value contains the field separator (,/;). If you want all the incoming String to be enclosed (not typical), check Force the enclosure around fields.

Length mapping also works fine. If you set a Length > 0, the step switches to a fixed-width formatting even when Pad fields is OFF. The difference is that when Pad field is OFF, Hop still writes enclosures. When it's ON, it doesn't write enclosure because it doesn't make sense since we know the start and end of that field. To me, this doesn't seem like a bug.

S3tuit avatar Sep 20 '25 13:09 S3tuit