json icon indicating copy to clipboard operation
json copied to clipboard

Don't consume token when returning `error.InvalidType`.

Open LordMZTE opened this issue 3 years ago • 2 comments

Sometimes, it's necessary to deserialize data where the type is uncertain. I tried doing this by first calling deserializeString, and if that returns an InvalidType error, I call deserializeMap. This doesn't work tho, since deserializeString will consume a token from the TokenStream, making it basically unusable afterwards.

LordMZTE avatar Jun 11 '22 09:06 LordMZTE

This would be very helpful, indeed now that I think about it.

However, Serde's able to avoid consuming tokens upon failure because it has a peek operation in its JSON parser. But as far as I know, there's no way to peek at the next token with std.json.TokenStream. The only ways I can think of are:

  • Make a copy of the tokens, which we can use later on to restore the tokens.
    • Getty JSON already does this in a few places, but I don't really like it. I can't remember if the copy was expensive or not, but it just looks/feels gross.
  • Add a peek method to std.json.TokenStream.
    • This would be ideal imo. But std.json's parser scares me so I haven't looked too deeply into this.
  • Implement parsing ourselves, like Serde does.
    • Obviously, this would be a lot of work. And if possible, I would like to avoid changing parsers until everything else in the library is relatively stable.

phan-do avatar Jun 21 '22 03:06 phan-do

update: A PR for a peek method in std.json has been made: https://github.com/ziglang/zig/pull/12798

phan-do avatar Sep 10 '22 04:09 phan-do