vscode-data-preview icon indicating copy to clipboard operation
vscode-data-preview copied to clipboard

Data conversion error from csv to DataPreview mode

Open gaille34 opened this issue 5 years ago • 7 comments

Hi, When I convert or visualize a .csv file with a value XX.0 in the first line of a column all values of the others lines of the same column loose the number after the point: -41.6 became -41. Please see the 2 attached screenshot. Best regards DataPreview_file csv_file

gaille34 avatar Nov 16 '20 15:11 gaille34

I hit a similar issue with this data.zip:

t,x1,x2,c1
0.0,1.0,3.0,1.453712e-11
0.01,0.99932516,3.0006747,1.4536232e-11
0.02,0.99864984,3.0013502,1.4535234e-11
0.03,0.99797404,3.002026,1.4534291e-11
...

imagen

Line plots are broken for the x1 and x2 columns, the other ones are parsed correctly.

IlyaOrson avatar Jan 19 '21 18:01 IlyaOrson

that's because your first line has ints and data type is detected based on the first data line.

also, not an error. I'd call it incorrect data precision issue at best :)

RandomFractals avatar Jan 20 '21 00:01 RandomFractals

Ok I see better now. Could it be possible to configure/manage this "precision" manually ? Today I have to be cautious on large datasets and I need to put 43.001

image

to have this result:

image

Best regards

gaille34 avatar Jan 20 '21 09:01 gaille34

yeah, I need to see if I can include more data rows for detecting ints and decimals, and there is a separate ticket for custom data types that I plan to address at some point (#156)

RandomFractals avatar Jan 20 '21 13:01 RandomFractals

that's because your first line has ints and data type is detected based on the first data line.

also, not an error. I'd call it incorrect data precision issue at best :)

Not an error in the code, but certainly an error in methodology. I'd suggest implementing some random sampling to infer type / precision instead of naively using the first row. This tool is unusable for me because of this bug.

Meussdorffer avatar Feb 07 '21 16:02 Meussdorffer

@Meussdorffer ok! I am always open to suggestions from devs with 0 commits :)

Always a great look when you patronize your users for making legitimate suggestions.

Meussdorffer avatar Feb 08 '21 02:02 Meussdorffer