dataframe-js icon indicating copy to clipboard operation
dataframe-js copied to clipboard

[BUG] Dataframe loaded from file doesn't identify missing values

Open itstillon opened this issue 4 years ago • 0 comments

Describe the bug When data is loaded from a file locally, DataFrame treats all values as string types (empty string) due to which "missing values" are not differentiated/tracked, due to which fillMissingValues() doesn't work as expected.

However, this works when the DataFrame is prepared on the fly (using new DataFrame(...))

To Reproduce Steps to reproduce the behavior:

  1. Copy the content below to a file & save (for e.g. test.csv)
name,age
Adam,10
Amy,
  1. Run the code below (assuming DataFrame dependency is added):
const DataFrame = require('dataframe-js').DataFrame;

(async() => {
  const df = await DataFrame.fromCSV('test.csv');
  df.fillMissingValues(0).show();
})();

Expected behavior Second row should have been updated with 0 as the age.

Screenshots

Desktop (please complete the following information):

  • OS: Ubuntu

Additional context This works as expected when the data is prepared in-house, i.e.

Code:

const DataFrame = require('dataframe-js').DataFrame;

(async() => {
  const df = await new DataFrame([
    { name: 'Adam', age: 10 },
    { name: 'Amy' }
  ]);
  df.fillMissingValues(0).show();
})();

Output:

itstillon avatar Apr 21 '21 19:04 itstillon