lnd icon indicating copy to clipboard operation
lnd copied to clipboard

tlv: allow tlv streams to use large records

Open Crypt-iQ opened this issue 3 years ago • 3 comments

Changes the tlv package such that:

  • Decode can decode records > 65535 bytes
  • DecodeWithParsedTypes can decode records > 65535 bytes
  • Introduces DecodeP2P which cannot decode records > 65535 bytes
  • Introduces DecodeWithParsedTypesP2P which cannot decode records > 65535 bytes

This is necessary for future changes to lnd and for other projects that want larger record sizes. Untrusted input should always use the *P2P variants

Crypt-iQ avatar Jul 29 '22 20:07 Crypt-iQ

After this is in and tagged, we can update the call-sites to use either the p2p or non-p2p variant

Crypt-iQ avatar Aug 01 '22 17:08 Crypt-iQ

Why not update them here, or instead lave the default call be the P2P one and then have a new DecodeNonP2P etc?

Only because I am not a pro at git tagging so it is easier for me if I chunk it

cause is it not the case that with the current change, we by default now dont bound the size of untrusted input from p2p land? That way we can also keep the absurd record length unit test.

Since we didn't make lnd update to this new tlv change, the old behavior of limiting to 65kb is still used. Same for other projects that require a specific tlv package version

Other than that, LGTM! (as long as no caller of the package tries to pass in a size too big for BigSize... which is extremely unlikely since it can handle like 18 exabytes 🤓 )

We could require a max for the non-p2p case, but I wouldn't know a good value

Crypt-iQ avatar Aug 03 '22 15:08 Crypt-iQ

Since we didn't make lnd update to this new tlv change, the old behavior of limiting to 65kb is still used. Same for other projects that require a specific tlv package version

oh! good point! i forgot that the tlv package had its own go.mod. My bad!

ellemouton avatar Aug 04 '22 05:08 ellemouton

Ready to land once the release notes have been updated!

Roasbeef avatar Aug 23 '22 20:08 Roasbeef

added notes

Crypt-iQ avatar Aug 23 '22 21:08 Crypt-iQ

Oops, another merge requires this PR to be rebased.

guggero avatar Aug 24 '22 07:08 guggero

Needs one more rebase!

Roasbeef avatar Aug 31 '22 19:08 Roasbeef

rebased

Crypt-iQ avatar Sep 14 '22 21:09 Crypt-iQ