execution-spec-tests icon indicating copy to clipboard operation
execution-spec-tests copied to clipboard

feat(fw,docs): Add eof/t8n tools specs, fix JSON schema generation in docs

Open winsvega opened this issue 1 year ago • 7 comments

🗒️ Description

Specs for eof tool implementation,

🔗 Related Issues

✅ Checklist

  • [x] All: Set appropriate labels for the changes.
  • [ ] All: Considered squashing commits to improve commit history.
  • [ ] All: Added an entry to CHANGELOG.md.
  • [ ] All: Considered updating the online docs in the ./docs/ directory.
  • [x] Tests: All converted JSON/YML tests from ethereum/tests have been added to converted-ethereum-tests.txt.
  • [x] Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
  • [x] Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.

winsvega avatar May 09 '24 08:05 winsvega

Hey @winsvega, I revamped the specs and introduced the JSON schemas for all types that the tools are expected to parse or output, including examples for most JSON types.

Use mkdocs serve to see the changes and let me know what you think.

marioevz avatar Jun 11 '24 00:06 marioevz

I am getting lost now. there was a nice description with examples why 600 lines of py code, is it now making automatically from the code structures?

the auto docs gen is ok, but I am not the type of a person who could read that. to me my initial version was much preferable. basically a short example in console on how to use it is the best type of doc for me.

looks like the generated doc is about eof test parser, not the eof tool.

with t8n however need more examples on all the types. yes. hm

currentNumber": "1",
  "currentTimestamp": "1000",
  "currentRandom": "0",
  "currentDifficulty": "1",
  "currentBaseFee": "2",
  "currentExcessBlobGas": "3",
  "parentDifficulty": "4",
  "parentTimestamp": "5",
  "parentBaseFee": "6",
  "parentGasUsed": "7",
  "parentGasLimit": "8",
  "currentBlobGasUsed": "9",

this can also be hex, but not padded. "0x1", "0x2"
another tricky part here that depending on the fork, this fields could be different. geth report an error if currentBaseFee provided but fork is not supporting the field.

also we have a notation here that if currentDifficulty not set, then there must be parentDifficulty and parentTimestamp so the tool will calculate currentDifficulty itself to be used for difficulty tests.

same with bsefee

winsvega avatar Jun 12 '24 07:06 winsvega

the auto docs gen is ok, but I am not the type of a person who could read that. to me my initial version was much preferable. basically a short example in console on how to use it is the best type of doc for me.

Yes we now have auto-gen of these specs, I think it's important because with each fork we introduce new fields to the Environment, so it's easy to link implementers to this page for them to look at an example.

I really like the example parts of the new auto-gen doc, more than the parameters, I think the types are one of the most important aspects.

this can also be hex, but not padded. "0x1", "0x2"

Yes I think it should only be hex, not decimal, more so when the other fields like withdrawals already have hex numbers in their fields. If you're ok with this, I can update the environment class to not produce anything but unpadded hex numbers as you mention.

another tricky part here that depending on the fork, this fields could be different.

This could definitely be added to the schema, I think we could have something like this:

  currentBaseFee:
    anyOf:
    - pattern: ^([0-9]|[1-9][0-9]+)$
      title: number
      type: string
    - type: 'null'
    default: null
    title: Currentbasefee
    fork: London

Although this is not within the standard of the JSON schema: https://json-schema.org/understanding-json-schema/reference/annotations

marioevz avatar Jun 12 '24 18:06 marioevz

@winsvega Updated Environment to use hex number exclusively in the latest commit, and if you try mkdocs serve again, the types and examples are automatically updated by the auto-gen: image

marioevz avatar Jun 12 '24 18:06 marioevz

could we keep my original description of eof code verification tool too?

also about forks some t8n return errors if you provide cancun field on legacy fork. I don't think we can fully autogen all the docs. still would be nice to have written description too.

winsvega avatar Jun 19 '24 07:06 winsvega

at this point this not my pr anymore

winsvega avatar Oct 09 '24 08:10 winsvega

at this point this not my pr anymore

I think it's still valuable to merge this, I'll take a look into rebasing it at some point to have it merged, but I agree that it morphed too much maybe, sorry about that.

marioevz avatar Oct 09 '24 15:10 marioevz

@marioevz lets finish this.

winsvega avatar Apr 07 '25 10:04 winsvega