libipld icon indicating copy to clipboard operation
libipld copied to clipboard

Remove the legacy codec system

Open Stebalien opened this issue 3 years ago • 7 comments

Serde is significantly more usable, and supporting two codec systems is very confusing and a waste of time.

Stebalien avatar Nov 15 '22 19:11 Stebalien

Agreed. Though I'd like to get close to feature parity, so that people using the legacy codec can smoothly upgrade.

vmx avatar Nov 16 '22 10:11 vmx

Heya, @vmx asked me to comment on our usage of the codec approach. To be honest, my first thought was the same as yours @Stebalien but there are two things that I enjoy in the codec approach:

  • Codecs know how to extract links
  • We mostly use dag-cbor, but in some cases we also encode as raw

I would love for there to be one true way, and if serializing data as blocks was as simple as deriving Serialize/Deserialize in most cases, that would be awesome :+1:

cdata avatar Nov 28 '22 15:11 cdata

I agree that we definitely need a way to cleanly extract links. It shouldn't be too difficult to re-implement that with serde by just throwing away everything we don't care about, but the current method is more efficient.

Stebalien avatar Nov 28 '22 17:11 Stebalien

I recently learned about the DeserializeSeed trait. Maybe this could be used to collect links while deserializing?

Frando avatar Dec 02 '22 22:12 Frando

That's not even necessary, really. We can just deserialize into a "links collector" that:

  1. Collects links.
  2. Ignores everything else.

Stebalien avatar Dec 03 '22 00:12 Stebalien

Ah right, that should be possible too. When working with many small blocks, the current API allows to collect all links into a single container (without allocating a links container for each block), that's why I thought of DeserializeSeed.

Maybe there's two features at play even?

  • Only collect links without deserializing everything
  • Deserialize into a struct and collect all links, in a single pass

The latter might not matter much in practice though.

Frando avatar Dec 03 '22 18:12 Frando

  • Only collect links without deserializing everything

  • Deserialize into a struct and collect all links, in a single pass

The latter might not matter much in practice though.

Agreed. I'd start with the former one and add the latter only if it turns out to be useful. I prefer keeping APIs small and targeted at needs, rather then adding theoretically useful things.

vmx avatar Dec 05 '22 10:12 vmx