[FEATURE] includeEndRowDelimiter defaults to true

[jmealo]

Parsing or Formatting?

• [x] Formatting
• [ ] Parsing

Is your feature request related to a problem? Please describe. When using cat or pv to pipe several large fast-csv formatted CSV files into Postgres (or anything expecting delimited output) not having includeEndRowDelimiter: true can really sink a lot of time. At the boundary of each file that doesn't end in a newline; you'll have a parsing error that starts at the end of a valid row leading to confusing/unhelpful error messages. Unfortunately, with large file-size, slow block-storage and wide lines this can be harder to troubleshoot than you'd think. In the interest of saving others trouble/pain; I wanted to suggest/advocate for revisiting this defaulting to false.

Is it possible that it's less harmful/more helpful to default this to true? In my experience, an error from a blank/extra line is easier to troubleshoot/handle than a missing line delimiter. Thoughts?

Describe the solution you'd like Consider/discuss whether it's safer/better to default includeEndRowDelimiter: true

Describe alternatives you've considered Add a warning to the documentation or explain the default value.

Additional context Thanks for the library

[jmealo]

An additional use case I found (or hit this issue again, rather) is if you pipe streams from Fast-CSV into pg-copy-streams or similar, they will fail with a non-sensical error message until you flush with a newline.

[daedalus28]

👍 We're defaulting this to true in our wrapper over this lib, too.