csv icon indicating copy to clipboard operation
csv copied to clipboard

Fix: Allow \r in unquoted fields when row separator doesn't contain \r

Open jsxs0 opened this issue 7 months ago • 2 comments

Fixes #60

This has been bugging me for a while - the CSV parser was rejecting \r characters in unquoted fields even when the row separator was something completely different like \n or a custom separator.

For example, this would fail unnecessarily:

CSV.parse("field1,field\rwith\rcr,field3\n", row_sep: "\n")

The problem was in prepare_unquoted where we were hardcoding "\r\n" instead of checking what the actual row separator was.

What changed:

  • Now we only exclude characters that are actually part of the row separator
  • If your row separator is \n, then \r is allowed in unquoted fields
  • If your row separator is \r\n, then both \r and \n are still properly excluded
  • Quoted fields work exactly the same as before

Testing:

  • Updated the tests that were expecting the old behavior
  • Added comprehensive tests for different row separator scenarios
  • All existing tests still pass

This makes the parser more flexible while keeping it safe for the cases where \r should actually be restricted.

jsxs0 avatar Jun 05 '25 05:06 jsxs0

@kou I've made all the changes asked.

jsxs0 avatar Jun 05 '25 08:06 jsxs0

@kou Thank you. I have:

  1. Removed the duplicate test
  2. Moved the unique tests to test_general.rb
  3. Fixed the misnamed test
  4. Deleted the separate test file

jsxs0 avatar Jun 05 '25 09:06 jsxs0

Thanks.

kou avatar Jul 04 '25 20:07 kou