node-csvtojson icon indicating copy to clipboard operation
node-csvtojson copied to clipboard

German Letters not being Encoded/Decoded properly

Open shaunstone0 opened this issue 4 years ago • 4 comments

When using the package, I have attempted to take a csv files that contains german umlauts (ä, ü) and they are coming out in the json file as T�rkei�

The correct encoding should be UTF-8, however, the package somehow messes up the words.

Or, perhaps I am doing it wrong:

csvtojson({
    headers: ['label','value','poop'], 
    colParser: {
            'label':function(item){
                    return item.toLowerCase();
            },
           'poop': "omit" 
    }}).fromFile(filePath).subscribe((object, err) => {
        object.default = 'false'
        JSON.stringify(object, ["value", "label", "default"])
        
        
    }).then((object) => {
        console.log(object);
        fs.writeFileSync('nation.json', JSON.stringify(object,null, 4), 'utf-8')
    })

shaunstone0 avatar Apr 29 '21 13:04 shaunstone0

This can be closed. Its Because of Excel removing the BOM when saving the file as a .CSV. You need to make sure to save the file as CSV UTF-8.

shaunstone0 avatar Apr 29 '21 14:04 shaunstone0

I would not close this issue because I am facing the same problem. The fromFile method should support other encodings than UTF-8 IMHO.

Do you agree @Keyang ?

jfoclpf avatar Jan 09 '22 22:01 jfoclpf

I got it :)

const fs = require('fs')
const csv = require('csvtojson')

csv()
  .fromStream(fs.createReadStream('/path/to/file', { encoding: 'latin1' }))
  .subscribe((json) => {
    console.log(json)
  },
  (err) => {
    throw err
  },
  () => {
    console.log('success')
  })

this is indeed a great package :)

jfoclpf avatar Jan 09 '22 23:01 jfoclpf

@shaunstone0 did it work the solution?

Would you please close the issue?

jfoclpf avatar Mar 21 '22 08:03 jfoclpf