witnet-rust icon indicating copy to clipboard operation
witnet-rust copied to clipboard

Enable retrieval of binary buffers

Open guidiaz opened this issue 3 years ago • 1 comments

  • In http/s sources, binary contents are specified by setting headers defined within MIME-type spec (RFC-2045). While data is serialized to text format during transmission, at client's app level is received as binary content.
  • By enabling binary data at the Witnet protocol level, either hash() or toString() reducers would be available within data requests.

guidiaz avatar Sep 19 '22 08:09 guidiaz

The conversion from bytes to string is implemented here:

https://github.com/witnet/witnet-rust/blob/63920bd875563741850225dd28ce98849adad299/rad/src/lib.rs#L242-L248

We are using this method from the surf library, which only supports UTF8: https://docs.rs/surf/2.3.2/surf/struct.Response.html#method.body_string

There is a body_bytes method which returns the raw bytes: https://docs.rs/surf/2.3.2/surf/struct.Response.html#method.body_bytes

So we could add a check like:

  • If the first operator of the script expects bytes as the input, use body_bytes
  • Otherwise, use body_string

The only problem is that we cannot access the request headers from RADON to read the content type, so converting the bytes into a string later will only be possible if we know the encoding ahead of time when creating the script. Similarly, when converting string to bytes we need to specify an encoding somehow. But encodings other than UTF8 are not supported yet, so that can be another issue.

tmpolaczyk avatar Sep 19 '22 11:09 tmpolaczyk

Superseeded by #2331

Tommytrg avatar Apr 24 '23 09:04 Tommytrg