"IO error: Failed to create dir: Invalid argument" when having non-latin characters in folder name (only on windows)

[apolkingg8]

const rocksdb = require("rocksdb")
const path = require("path")

rocksdb(path.resolve(__dirname, '中文')).open(function (res) {
    debugger
})

IO error: Failed to create dir: G:\GitHub\pouchdown-chinese-test\中文: Invalid argument

Mac works fine, only happened on Windows.

[apolkingg8]

Tried leveldown, and it works.

const leveldown = require("leveldown")
const path = require("path")

leveldown(path.resolve(__dirname, '中文')).open(function (res) {
    debugger
})

[apolkingg8]

Found a similar issue in a rust-rocksdb binding. Maybe we can do something like this PR in node binding file for this issue?

[ralphtheninja]

Not exactly sure where the problem lies. Since it works on leveldown and rocksdb basically is a fork of that, there is either some differences in that layer or in the native lib itself. I took a quick glance at leveldown and it seems to me that we are just passing in a utf8-string to the C++ layer.

[apolkingg8]

I'm not a c++ programmer, but it looks like rocksDB use _mkdir to create dir in windows. I tried something after some searching:

string dir_name = "中文";
_mkdir(dir_name.c_str()); // wont work

string dir_name = "中文";
_wmkdir(utf8_to_utf16(dir_name).c_str()); // it's works!

hope that'll be helpful, sorry I'm not familiar with this part.

[ralphtheninja]

I'm right now wondering if the rocksdb lib should be compiled with Unicode enabled (UNICODE=true or similar) because I think that string is just a placeholder

[vweevers]

RocksDB decided not to fix this: https://github.com/facebook/rocksdb/issues/3408#issuecomment-403639956.

Tagging with help wanted; if someone wants to take a stab at this we'll gladly review a PR.

[vweevers]

Update (https://github.com/facebook/rocksdb/issues/3408#issuecomment-513957920):

This is now fixed in https://github.com/facebook/rocksdb/pull/4469 without having to use a custom env by using the WITH_WINDOWS_UTF8_FILENAMES option which defines ROCKSDB_WINDOWS_UTF8_FILENAMES.

So we can tackle this once https://github.com/Level/rocksdb/pull/143 lands.