elements icon indicating copy to clipboard operation
elements copied to clipboard

PSET-v2 Output Serialisation

Open sekulicd opened this issue 4 years ago • 8 comments

If i understood correctly keypairs should be serialised in order as defined by BIP 174 and in [here].(https://github.com/ElementsProject/elements/blob/master/doc/pset.mediawiki)

Currently this is not the case for output serialisation as script is serialised before amount in here. As per BIP 174 it should be opposite.

I can see @achow101, in PSBT2 branch, follows order defined in BIP 174/370, check here.

sekulicd avatar Oct 19 '21 09:10 sekulicd

BIP 174 recommends a serialization order, but records are not required to be in that order. Your deserializer must be able to handle records in a different order and should not rely on records being provided in a specific order.

achow101 avatar Oct 19 '21 16:10 achow101

Yes i guess deserialisation is not impacted, but since PSET serialisation is not standardised then round trip test defined in here could fail(like in my case) for different serialisation implementation since input byte array(before deserialised to PSET object) doesnt need to be same as output byte array.

sekulicd avatar Oct 20 '21 11:10 sekulicd

Yes, that unit test is specifically for rust-elements (which makes up its own canonical serialization, specifically so that we could write round-trip tests for our fuzzer).

apoelstra avatar Oct 28 '21 21:10 apoelstra

BIP 174 recommends a serialization order, but records are not required to be in that order. Your deserializer must be able to handle records in a different order and should not rely on records being provided in a specific order.

Is the serialization order defined in BIP 174 actually a recommendation? Consider this, taken from the test vectors, for instance:

A creator creating a PSBT for a transaction which creates the following outputs:

  • scriptPubKey: 0014d85c2b71d0060b09c9886aeb815e50991dda124d, Amount: 1.49990000
  • scriptPubKey: 001400aea9a2e5f0f876a588df5546e8742d1d87008f, Amount: 1.00000000

and spends the following inputs:

  • TXID: 75ddabb27b8845f5247975c8a5ba7c6f336c4570708ebe230caf6db5217ae858, Index: 0
  • TXID: 1dea7cd05979072a3578cab271c02244ea8a090bbb46aa680a65ecd027048d83, Index: 1

must create this PSBT:

  • Bytes in Hex:
70736274ff01009a020000000258e87a21b56daf0c23be8e7070456c336f7cbaa5c8757924f545887bb2abdd750000000000ffffffff838d0427d0ec650a68aa46bb0b098aea4422c071b2ca78352a077959d07cea1d0100000000ffffffff0270aaf00800000000160014d85c2b71d0060b09c9886aeb815e50991dda124d00e1f5050000000016001400aea9a2e5f0f876a588df5546e8742d1d87008f000000000000000000
  • Base64 String:
cHNidP8BAJoCAAAAAljoeiG1ba8MI76OcHBFbDNvfLqlyHV5JPVFiHuyq911AAAAAAD/////g40EJ9DsZQpoqka7CwmK6kQiwHGyyng1Kgd5WdB86h0BAAAAAP////8CcKrwCAAAAAAWABTYXCtx0AYLCcmIauuBXlCZHdoSTQDh9QUAAAAAFgAUAK6pouXw+HaliN9VRuh0LR2HAI8AAAAAAAAAAAA=

And similar ones are defined for other roles (updater, signer, combiner). The serialization order defined MUST be preserved so, even if it would be totally fine to follow a different one, considering it would then produce hex/base64 string different from those in the BIP, should it be actaully valid?

altafan avatar Nov 04 '21 11:11 altafan

cc @achow101

altafan avatar Nov 15 '21 16:11 altafan

@altafan

The serialization order is a recommendation. The test vectors are for software that follow the recommendations. It is difficult to write test vectors that use different ordering while also enforce that the correct fields are added.

achow101 avatar Nov 15 '21 17:11 achow101

@achow101, what would be recommended workflow for testing whether the pset/psbt created by elementsd and the one parsed by another implementation are the same?

sanket1729 avatar Dec 08 '21 04:12 sanket1729

@sanket1729 Generally it would be to have both implementations output their internal representations and compare that the internal representations match each other. Obviously this cannot be generalized and will need to be specific for each implementation. For example, you could use decodepsbt and check that the data output by the decoding match what is being stored internally by the other implementation (e.g. compare that the hex of a pubkey matches the bytes of supposedly the same pubkey).

achow101 avatar Dec 08 '21 05:12 achow101