elixir-nodejs icon indicating copy to clipboard operation
elixir-nodejs copied to clipboard

Broken encoding

Open utopos opened this issue 4 years ago • 2 comments

Hi!

I have been trying the library (which is fantastic!) but I'm experiencing issues with characters encoding. If the return string from the called JS function contains UTF-8 characters, the return on Elixir side will be a binary.

Please, allow me to illustrate with en example with a letter "Ł":

Javascript:

module.exports = function (text){
    return text
}

Elixir:

iex> {:ok, x} = NodeJS.call("test",["hełło"])
iex> String.codepoints(x)
["h", "e", "Å", <<194, 130>>, "Å", <<194, 130>>, "o"]

Thank you!

utopos avatar Oct 15 '21 03:10 utopos

Created a PR here https://github.com/revelrylabs/elixir-nodejs/pull/77

29decibel avatar Jan 17 '23 00:01 29decibel

From the README:

In order to cope with Unicode character it is necessary to specify the binary option:

NodeJS.call("echo", ["’"], binary: true) # => {:ok, "’"}

Nicolab avatar Feb 06 '24 12:02 Nicolab

Hey, @29decibel thanks for filing the issue and PR, and sorry it's been sitting here so long without attention. If you're still interested in moving this forward, I'd be interested in your take on a couple of things:

  • It does appear that the binary: true option addresses the example in the issue and the test case in your PR... are you aware of other cases that aren't addressed by the binary: true option?
  • Or is the idea here just that we could eliminate the need for the binary: true option by always passing the response through IO.iodata_to_binary()?

If the latter is the case, then it seems like something that'd make the library simpler and easier to use, though we'd want to be careful not to break other use cases that are currently working fine.

grossvogel avatar May 22 '24 22:05 grossvogel

Thank you @grossvogel ! I missed this option then, I think we can close this PR. :-)

29decibel avatar May 22 '24 22:05 29decibel

@utopos I'm going to close this under the understanding that the binary: true option solves your needs. If that's not the case, please let me know!

grossvogel avatar May 23 '24 13:05 grossvogel