Document how to add a bytecode specialization

[Fidget-Spinner]

How to add a new bytecode specialization is knowledge only a few of us remember right now. We should document this somewhere (devguide or CPython, I don't mind). Hopefully a new contributor can pick this up!

There reference is this PR: https://github.com/python/cpython/pull/116385/files

Assuming you found an instruction that serves as a good specialization candidate. Let's use the example of CONTAINS_OP:

1. Set the original CONTAINS_OP to a uop by changing it from CONTAINS_OP to _CONTAINS_OP in Python/bytecodes.c. The instruction definition should also change from inst to op.
2. Add the uop that calls a specializing function. In this case _SPECIALIZE_CONTAINS_OP.
3. The original CONTAINS_OP is now a new macro consisting of _SPECIALIZE_CONTAINS_OP + _CONTAINS_OP.
4. Define the cache structure in Include/internal/pycore_code.h. At the very least, a 16-bit counter is needed.
5. Write the specializing function itself in Python/specialize.c. Refer to any other function in that file for the format.
6. Remember to update operation stats by calling add_stat_dict in Python/specialize.c.
7. Add the cache layout in Lib/opcode.py so that Python's dis module will know how to represent it properly.
8. Bump magic number in Include/internal/pycore_magic_number.h.
9. Run make regen-all on *nix or build.bat --regen on PC.

[AbudiMutamba]

Hey I would love to work on this issue if you don't mind, thanks,

[willingc]

Hi @AbudiMutamba. Welcome :wave: Before you get too far along on this. Let's check with @Fidget-Spinner if this is still something that is needed?

[Fidget-Spinner]

Yes we still need this.

[willingc]

Thanks @Fidget-Spinner. If you want to give this a try @AbudiMutamba, please do so.

[faizanoor3001]

Hello, @willingc is it something still open ? Would like to contribute. :)

[Fidget-Spinner]

@faizanoor3001 this is still open

[faizanoor3001]

Hello, @Fidget-Spinner, I’ve opened a PR adding “How to Add a New Bytecode Specialization” to the How-To section. We can verify the text changes or if any updates are required, but I’m unsure about the best place for this new guide. Would it fit better in:

1. Interpreter.md section - appended under the existing InternalDocs?

2. CPython Internals — as its own subsection?

   Current:

   Potential Alternative:

   

PS: This is my first contribution, so I appreciate your guidance. I’m also studying the Crafting Interpreters book, and your original PR helped me delve deeper into the interpreter’s workings. Thank you!

[Fidget-Spinner]

@faizanoor3001 thanks for taking this up! I will review it soon.

[faizanoor3001]

@Fidget-Spinner Raised a PR, but since the issue is in devguide project and the PR I have raised in the cpython project, the bot is unable to recognize the gh-NNNN issue number in this case gh-1286 as correct issue number, could you please help here, as I guess we would need a new issue number ? Or how to work around on this ?