dsq icon indicating copy to clipboard operation
dsq copied to clipboard

Newline in xlsx header causes crash

Open infojunkie opened this issue 3 years ago • 5 comments

Using the attached xlsx sheet, running dsq newline.xlsx --schema generates the following error:

json: error calling MarshalJSON for type *runner.Shape: invalid character '\n' in string literal

Running dsq newline.xlsx does not crash:

[{"Header with \nNewline":"1"},
{"Header with \nNewline":"2"},
{"Header with \nNewline":"3"},
{"Header with \nNewline":"4"}]

Removing the newline from the header cell no longer crashes the schema command.

infojunkie avatar Jun 26 '22 04:06 infojunkie

This may also be related to #79 and https://github.com/multiprocessio/dsq/releases/tag/0.20.2

eatonphil avatar Jun 26 '22 14:06 eatonphil

Tested this out and it's its own issue. I think I'll deal with this be dropping all newlines in column names for all file types since they all seem to break SQLite.

eatonphil avatar Jun 26 '22 18:06 eatonphil

With the help of Twitter I figured out a workaround :D

dsq newline.xlsx "$(printf "select [Header with \nNewline] from {}")"
image

eatonphil avatar Jun 26 '22 19:06 eatonphil

The issue is that bash won't insert a literal newline but other tools like printf can do that.

eatonphil avatar Jun 26 '22 19:06 eatonphil

Thanks for the workaround! Yes, dropping the newline would simplify handling.

infojunkie avatar Jun 26 '22 22:06 infojunkie