support japanese character

[dinh-duc-nghia-p]

hi developer team

i cannot set encoding: 'SJIS' for createCsvWriter function, please support me !

[ryu1kn]

Hi @dinh-duc-nghia-p , sjis is not one of node.js's supported encodings. Do you want to provide us more specific info as to what you want to achieve? Node.js supports utf8 and you can write Japanese characters with it. If this doesn't fit for your situation, is that because the data you want to write to a file is encoded with sjis or you actually want to write a CSV file with sjis encoding?

[dinh-duc-nghia-p]

@ryu1kn thanks for support. i actually want to export one file with sjis encoding. in japan we often use sjis encoding for csv file.

[ryu1kn]

I guess you need to use a thrid-party library like iconv-lite to write with sjis. I can probably introduce a new parameter or make encoding to accept a function for custom encoding. Something like:

const csvWriter = createObjectCsvWriter({
    path: 'path/to/file.csv',
    header: [
        {id: 'name', title: 'NAME'},
        {id: 'lang', title: 'LANGUAGE'}
    ],
    encoding: text => iconv.encode(text, "Shift_JIS")
});

[ryu1kn]

But for now, if you use csv-writer, you need to do the file write part yourself...

const {writeFile} = require('fs').promises
const iconv = require('iconv-lite')
const createCsvStringifier = require('csv-writer').createObjectCsvStringifier

const toShiftJIS = text => iconv.encode(text, "Shift_JIS")

const csvStringifier = createCsvStringifier({
  header: [
    {id: 'name', title: '名前'},
    {id: 'lang', title: '言語'}
  ]
})

const records = [
  {name: 'ボブ',    lang: 'フランス語, 英語'},
  {name: 'メアリー', lang: '英語'}
]

const headerLine = csvStringifier.getHeaderString()
const recordLines = csvStringifier.stringifyRecords(records)

writeFile('foo.csv', toShiftJIS(headerLine + recordLines))

[ryu1kn]

Or using stream like this:

const {createWriteStream} = require('fs')
const {Readable, Transform} = require('stream')
const iconv = require('iconv-lite')
const createCsvStringifier = require('csv-writer').createObjectCsvStringifier

toShiftJIS = text => iconv.encode(text, "Shift_JIS")

class PeopleTransformStream extends Transform {
    constructor() {
        super({objectMode: true})

        this._firstRecord = true
        this._csvStringifier = createCsvStringifier({
            header: [
                {id: 'name', title: '名前'},
                {id: 'lang', title: '言語'}
            ]
        })
    }

    _transform(person, _encoding, callback) {
        const sjisCallback = text => callback(null, toShiftJIS(text))
        const personLine = this._csvStringifier.stringifyRecords([person])
        if (this._firstRecord) {
            this._firstRecord = false
            sjisCallback(this._csvStringifier.getHeaderString() + personLine)
        } else {
            sjisCallback(personLine)
        }
    }
}

const transformStream = new PeopleTransformStream()
const writeStream = createWriteStream('foo.csv')

const records = [
  {name: 'ボブ',    lang: 'フランス語, 英語'},
  {name: 'メアリー', lang: '英語'}
]

Readable.from(records)
  .pipe(transformStream)
  .pipe(writeStream)

[sg552]

Yes, we need this feature! Chinese people also need the 'GBK' encoding for writing csv on Windows.

I guess you need to use a thrid-party library like iconv-lite to write with sjis. I can probably introduce a new parameter or make encoding to accept a function for custom encoding. Something like:

const csvWriter = createObjectCsvWriter({
    path: 'path/to/file.csv',
    header: [
        {id: 'name', title: 'NAME'},
        {id: 'lang', title: 'LANGUAGE'}
    ],
    encoding: text => iconv.encode(text, "Shift_JIS")
});