`chain-spec`: support for `json` config/patch (and `GenesisBuilder` API)

[michalkucharczyk]

This PR:

• adds support for:
  • JSON based GenesisConfig to ChainSpec allowing interaction with runtime GenesisBuilder API.
  • interacting with arbitrary runtime wasm blob to chain-spec-builder command line util,
  • GenesisBuilder API in kitchensink runtime
• removes code from system_pallet
• adds code to the ChainSpec
• deprecates ChainSpec::from_genesis, but also changes the signature of this method extending it with code argument. ChainSpec::builder() should be used instead.
• implements GenesisBuilder API for node-template-runtime and kitchensink-runtime.

---

code moved into ChainSpec

Having the explicit code field in ChainSpec would allow for duplication of the wasm code within the raw version of the ChainSpec (as the raw storage may contain wasm code under genesis::top::raw::0x3a636f6465 key).

To avoid this redundancy/ambiguity following measure was implemented. When re-exporting the raw version of ChainSpec the code field will be converted into genesis::top::raw::0x3a636f6465. It will be removed from the ChainSpec. If the raw ChainSpec already contains genesis::top::raw::0x3a636f6465 entry, it will be overwritten with the value of the code field (if present).

Similarly, when building genesis state (actually calling ChainSpec::assimilate_storage) the existing 0x3a636f6465 will also be overwritten with code field (if present).

Example:

{
  "name": "TestName",
  "id": "test_id",
  "chainType": "Local",
   ...
  "genesis": {
    "raw": {
      "top": {
        "0x3a636f6465": "0x010101"
      },
      "childrenDefault": {}
    }
  },
  "code": "0x060708"
}

is equivalent of:

{
  "name": "TestName",
  "id": "test_id",
  "chainType": "Local",
   ...
  "genesis": {
    "raw": {
      "top": {
        "0x3a636f6465": "0x060708"
      },
      "childrenDefault": {}
    }
  }
}

JSON based GenesisConfig in ChainSpec:

Patch

The ChainSpec can now be built using genesis config JSON patch (which contains some key-value pairs meant to override runtime's genesis config default values). This can be achieved with with_genesis_patch method of the builder:

let chain_spec = ChainSpec::builder()
    .with_code(substrate_test_runtime::wasm_binary_unwrap())
    .with_genesis_config_patch(json!({
        "babe": {
            "epochConfig": {
                "c": [
                    7,
                    10
                ],
                "allowed_slots": "PrimaryAndSecondaryPlainSlots"
            }
        }
    }))
    .build()

Resulting ChainSpec instance can be converted to raw version of chain spec JSON file. This was not changed and can be done with chain_spec.as_json(true) method. Sample output is here. The runtime's GenesisBuilder::build_config API is called during this conversion.

The ChainSpec instance can also be written to chain spec JSON file in human readable form. The resulting chain spec file will contain the genesis config patch (partial genesis config). Sample output is here

Full Config

It is also possible to build ChainSpec using full genesis config JSON (containing all the genesis config key-value pairs). No defaults will be used in this approach. The sample code is as follows:

let chain_spec = ChainSpec::builder()
    .with_code(substrate_test_runtime::wasm_binary_unwrap())
    .with_genesis_config(json!({
        "babe": {
            "authorities": [
                [ "5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY", 1 ],
                ...
            ],
            "epochConfig": {
                "allowed_slots": "PrimaryAndSecondaryPlainSlots",
                "c": [ 3, 10 ]
            }
        },
        "balances": {
            "balances": [
                [ "5D34dL5prEUaGNQtPPZ3yN5Y6BnkfXunKXXz6fo7ZJbLwRRH", 100000000000000000 ],
            ],
            ...
        },
        "substrateTest": {
            "authorities": [
                "5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY",
                ...
            ]
        },
        "system": {}
    }))
    .build()

Again, resulting ChainSpec instance can be converted to the raw version of chain spec JSON file (which involves calling GenesisBuilder::build_config runtime method) . It can be also stored in human readable version, sample output here.

chain-spec-builder util

New commands allowing to interact with arbitrary WASM runtime blob were added. Use cases are as follows:

Get default config from runtime

Queries the default genesis config from the provided runtime.wasm and uses it in the chain spec. Can also store runtime's default genesis config in given file (-d):

chain-spec-builder runtime -r runtime.wasm default -d /dev/stdout

Note: GenesisBuilder::create_default_config runtime function is called.

Generate raw storage chain spec using genesis config patch

Patches the runtime's default genesis config with provided patch.json and generates raw storage (-s) version of chain spec:

chain-spec-builder runtime -s -r runtime.wasm patch -p patch.json

Note: GenesisBuilder::build_config runtime function is called.

Generate raw storage chain spec using full genesis config

Build the chain spec using provided full genesis config json file. No defaults will be used:

chain-spec-builder runtime -s -r runtime.wasm full -c full-genesis-config.json

Note: GenesisBuilder::build_config runtime function is called.

Generate human readable chain spec using genesis config patch

chain-spec-builder runtime -r runtime.wasm patch -p patch.json

Note: No runtime API is called.

Generate human readable chain spec using full genesis config

chain-spec-builder runtime -r runtime.wasm full -c full-genesis-config.json

Note: No runtime API is called.

Some extra utils:

• verify: allows to verify if human readable chain spec is valid (precisely: all required fields in genesis config are initialized),
• edit, allows to:
  • update the code in given chain spec,
  • convert given chain spec to the raw chain spec,

Some open questions/issues:

• ~~naming .with_no_genesis_defaults + in chain spec json keys: JsonPatch / JsonFull,~~
• ~~GenesisSource source for patch and full config~~.
• support for New/Generate commands in `chain-spec-builder? (IMO we can remove them).

Step towards: paritytech/polkadot-sdk#25

polkadot companion: paritytech/polkadot#7508 cumulus companion: paritytech/cumulus#2936

[michalkucharczyk]

bot rebase

[michalkucharczyk]

bot rebase

[michalkucharczyk]

bot fmt

[michalkucharczyk]

bot clean

[michalkucharczyk]

bot fmt

[michalkucharczyk]

bot rebase

[michalkucharczyk]

bot clean