Metadata parameters removed from events

[bedeho]

Background

It turns out that Substrate stores events in storage, and this has some costs

https://substrate.stackexchange.com/questions/4126/why-are-events-stored/4132#4132

Proposal

• Runtime: Remove metadata parameters from events, e.g. conent, membership, remarks, etc.
• QN: Process extrinsics rather than events, or alternatively, work back from event to extrinsic to recover metadata field (probably way easier). It would be really really really! good if we could avoid processing extrinsics, because trying to process metatransactoins like sudo,batch, sudoas etc., in order to figure out what happened, will just become a nightmare. We really need to map out how well this would work across all QN mappings before we dare do anything

Its not obvious this is mainnet must-have.

┆Issue is synchronized with this Asana task by Unito

[dmtrjsg]

@bedeho based on the recent outlook is it likely to be added to mainnet scope?

[bedeho]

I honestly don't know here, but it should at worst have QN implications, not any higher up in the stack. I'm pretty sure that, doing this later does not cause any problems, if the fix is actually feasible, as new QN mappings would still work for old events which do indeed have the metadat payload, so you should be able to just resynch from genesis and everything working, but we should double-check this. Is this true @zeeshanakram3 ?

If its not ,then one has to consider doing this before launch, but I am not 100% how this impacts the QN if we do the change later and I am wrong about the backwards compatible synching.

[zeeshanakram3]

Process extrinsics rather than events, or alternatively, work back from event to extrinsic to recover metadata field (probably way easier)

I think its possible to work back and extract payload parameters from extrinsic. Also, we can just create some new interfaces/classes in Hydra that expose payload data to the mappings, similar to how events data is available in mappings e.g. for ChannelCreated mapping

async function content_ChannelCreated({ store, event }: EventContext & StoreContext): Promise<void> {
  // currently hydra generated code exports `ChannelCreatedEvent` class that mappings use
  // to get the event values. In similar way, we could add support for extrinsic' payload from given 
  // event, e.g. create `ChannelCreatedPayload` class for exposing the payload params
  const [channelId, channel, channelCreationParameters, rewardAccount] = new Content.ChannelCreatedEvent(event).params

  // TODO: read payload data 
  const [, channelCreationParameters] = new Content.CreateChannelPayload(event).params
}

the fix is actually feasible, as new QN mappings would still work for old events which do indeed have the metadat payload, so you should be able to just resynch from genesis and everything working.

Also, the nature of change we intend to make in mappings wont make new mappings incompatible with the older ones if runtime events still have metadata param.

I will try to do a POC over the weekend to know the feasibility and validate the assumption

[zeeshanakram3]

I had a sneak peek at the Hydra for implementing the feature for extracting extrinsics payload parameters provided the event. And the good news is that Hydra already has support for that.

For any given extrinsic extrinsic_name, hydra exports class ExtrinsicNameCall that contains the information about extrinsics parameters, e.g., for create_channel extrinsic, we can use CreateChannelCall to get the payload values. Also, I tested this feature for a few extrinsic(create_channel, member_remark, etc.) by using the metadata values from the payload instead of an event, and it's working fine.

[dmtrjsg]

@zeeshanakram3 great news, thanks a mil for bringing the good news 🙌

[bedeho]

But what happens if the final extrinsic extrinsic_name is actually invoked by a top level extrinsic like sudo,batch, sudoas ? I don't think this lookup using ExtrinsicNameCall will refer to extrinsic_name, it will refer to these top level once, and your logic will break, right?

If so, then you will need to make the mappings much more complex, by having them try each of the following

• create_channel
• sudo(create_channel(metadata))
• sudo_as(...create_channel(metadata))
• possibly other top level calls?

and actually, isn't it even wors? because some nut-job can start composing these? so batch(batch .....reate_channel(metadata)

It's not clear hot to deal with this at all, or certainly in a good way, and in particular across all the different extrinsic which have metadata. 🤔

[bedeho]

Turns out to not be needed.