python
Add a Sphinx role to link to GitHub files
This PR adds a role to link to GitHub files (in the python/cpython repo) and uses them in internals/parser.rst.
- Fixes #960
One idea, filenames are often written with a fixed-width font (e.g. in this style guide), shall we have the role do that too?
It took me a bit but eventually (and with the help of @AA-Turner and @CAM-Gerlach) I figured out how to make it a literal, in a way that is simple and concise enough.
While I was at it, I also added support for ! so that we can do e.g. :cpy-file:`!Makefile` without creating a link (since Makefile is generated, and not in the actual repo).
I also thought about adding support for variables parts like :cpy-file:`somedir/python3.{x}/somefile.py`, but this seems more involved and IMHO not worth it. If there is a variable part, the file can't be linked anyway and in that case a regular :file: can be used (even if it's slightly less semantic in the source, but otherwise unnoticeable since they render the same).
Support for ~ (to strip the dirs and leave the filename) seems YAGNI to me, but it's not too difficult to add if we really wanted to.
This is how the output of the current PR looks like:
And document this role at https://cpython-devguide--961.org.readthedocs.build/documentation/markup.html#roles ?
Those roles are just for the cpython repo so I don't think this should be documented. If we end up adding it to python/cpython (and python/peps too), then it can be documented. (Doing that would also mean duplicating (or triplicating) the code, so we might want to come up with a better solution.)
Note that cpython already defines a :source: role (which is also not documented).
Click to see some related findings
Sphinx defines a bunch of extra nodes, including a literal_emphasis, but it doesn't support variables parts and just makes everything italic. I haven't seen a node that already does what we need:
https://github.com/sphinx-doc/sphinx/blob/6ed4bbba396eb410c4d00e5f9881aa621cec749a/sphinx/addnodes.py#L512
A new node type that combines literal and reference could be created and used as well:
https://github.com/docutils/docutils/blob/c5a6ec93fe81fc44851f65aff37bef8a4cd2d173/docutils/docutils/nodes.py#L1893-L1894
Sphinx also defines a EmphasizedLiteral role (which is what is used for the :file: role):
https://github.com/sphinx-doc/sphinx/blob/6ed4bbba396eb410c4d00e5f9881aa621cec749a/sphinx/roles.py#L272
If we wanted to handle this at the role level (rather than the node level) we could subclassed that and enhance it to support linking. This shouldn't be too difficult, but it's not entirely obvious to me how to do it and I'm happy with the current solution.
Ah, I didn't realize autolink was a custom role defined for this repo (I'd seen it mentioned being used on the NumPy repo and elsewhere); my assumptions and a few comments I'd seen when googling had led me to believe otherwise, and we'd have to re-implement it ourselves (in which case I went with the class approach), but don't know why I didn't double check.
In any case, yeah, its simple to add that given it's already defined here, especially for a first implementation. As for
If we end up adding it to python/cpython (and python/peps too), then it can be documented. (Doing that would also mean duplicating (or triplicating) the code, so we might want to come up with a better solution.)
I've drafted (but not yet tested, so it likely needs some debugging) a common base class that provides a superset of the existing functionality, including everything here plus customizable branches/tags/commits, ~ support, custom link titles (which are automatically not rendered as a literal), emulating the CSS classes of the file role, and some other things, while being fully extensible to files on any GitHub project, or indeed any arbitrary link.
It needs some more refactoring (to move the default org/repo to config settings rather than a custom subclass, and provide a factory function to generate new ones), a couple more features (support for fragments and validation so it works with PEPs/RFCs, support for GitHub issues/PRs), and a few tweaks (using @ instead of : for branches/tags/commits, adding URL escaping). However, it could form a common base class (and subclasses) implemented in a Sphinx extension to provide many of the similar, currently-custom roles used/desired across our repos (GitHub issue/PR, BPO issue, PEPs, and RFCs, alongside source files, which could all specify other repos or even orgs if needed), with a superset of the existing functionality, to replace the patchwork of repo-bespoke code. Going forward, building off this would likely make more sense than copying things around piecemeal between repos, and would be easily reusable for other projects as well.
If we wanted to handle this at the role level (rather than the node level) we could subclassed that and enhance it to support linking. This shouldn't be too difficult, but it's not entirely obvious to me how to do it and I'm happy with the current solution.
That's quite possible, but would require basically re-implementing the role, which is really just the same as :samp: with an different CSS class, as it is basically one big monolithic function and almost all the logic is devoted to parsing the sample bits, which aren't useful for a concrete specific file (that this role represents) as opposed to an abstract file that the file role corresponds to. Your approach here is far more reasonable, especially as a first implementation.
Longer-term, the above proposal features a common base class with many small, overridable methods, which can be easily subclassed used for other types of links and custom behavior, which avoids having to reimplement everything.
Ah, I didn't realize
autolinkwas a custom role defined for this repo (I'd seen it mentioned being used on the NumPy repo and elsewhere);
Yeah, it's one of those handy bits of sample code that gets copied and pasted around a lot!
I'm going to merge this and follow up with another PR to use it with other files. We can then improve on it with other PRs and possibly port it to cpython and the peps repos.