simd-lite
How is the library memory consumption compared to serde_json?
Looking around it seems like serde_json memory overhead is significant I wonder how good is simd-json compared to it. Also it seems like there are other libraries that are trying to improve it: https://github.com/Diggsey/ijson
That's a good question, I am fairly sure worse - memory hasn't been the primary focuses and simdjson does prioritize throughput over memory, actively trading in more memory usage for higher performance. We use a tape that gets created as we parse and then collect it into value or struct, as well as a scratchpad string buffer so we don't can avoid allocations. So all in all my guess is worse.
As for the Value, not sure, it depends on which, the owned::Value, probably about the same, the borrowed::Value probably a bit better (as it re-uses the input memory for strings and avoids allocating them).
iJSON looks really interesting so :) there might a few tricks in there worth adopting!
https://github.com/Diggsey/ijson/issues/7#issuecomment-987747154 The question is can we "merge" both libraries somehow to improve both memory and cpu?
Sorry for the late reply, got a bit distracted with pre-vacation wrapup. I think it would be very easy to allow for the tape to be translated to the a ijson value, or even nicer implement the value-trait for it (that would would take care of 99% of it)
With the latest update simd-json allows completely allocation free parsing when used carefully, this is tested to guarantee the constraint is kept.
This obviously won't apply to every use but it allows users to control their experience.