idefix icon indicating copy to clipboard operation
idefix copied to clipboard

MNT: compile (pin) doc build dependencies

Open neutrinoceros opened this issue 1 year ago • 9 comments

This is a first step towards #191 (missing the automated updates for now, see below)

Notes:

  • I created a distinction between the set of direct dependencies (requirements.in) and the complete set of dependencies (direct dependencies + their dependencies + ...) which are requirements*.txt files with everything pinned (except the Python runtime version, but that's done in .readthedocs.yaml), ususally refered to as a "compiled" requirement file
  • compiled requirements can be obtained with either of the following commands (either can also be used to upgrade requirements later)
    • uv pip compile requirements.in (via uv) (this is the tool I'm using, for reference)
    • pip-compile requirements.in (via pip-tools)
  • seeing that direct dependencies were initially pinned in python_requirements.txt (renamed to requirements.in), I produced two set of compiled requirements:
    • one with this constraint still applied (this is requirements_direct_pinned.txt)
    • one with == replaced by >= (this is requirements.txt)

I would recommend using the later (if it works), but the former is probably closer to what has been running on RTD as of late so it can serve as a fallback if the most up-to-date set doesn't fly.

I'm also keeping auto-updates via dependabot for a follow up PR because I want to first know which version of compiled requirements should be kept. This might be a little tricky to test: as far as I'm aware, RTD only runs once a patch is merged (is this right ?). Also, I'm assuming RTD runs on commits made to master, not develop, so maybe I should target master with this PR ?

neutrinoceros avatar Dec 17 '24 08:12 neutrinoceros

Thanks @neutrinoceros . However, I must say I don't know how to test this... and I'm not keen on merging this straight away onto master.

glesur avatar Dec 19 '24 07:12 glesur

I should be able to build docs locally with both files and send you the artifacts if you'd like.

neutrinoceros avatar Dec 19 '24 07:12 neutrinoceros

So actually it's pretty simple: requirements_direct_pinned.txt is the only alternative that works at the moment. I'll replace requirements.txt with it.

neutrinoceros avatar Dec 19 '24 08:12 neutrinoceros

I you'd like, I could add a CI job that mimicks what RTD does, so you don't have to rely on local/manual testing everytime something is upgraded. I think it's also possible to configure RTD to build on pull requests (without publishing the result, of course), but that's out of my hands.

neutrinoceros avatar Dec 19 '24 08:12 neutrinoceros

I think that we definitely need a CI job to check the doc. I can have a go at this (or possibly @lih ). Let me know if you start something.

glesur avatar Dec 19 '24 08:12 glesur

I don't think I will have time to work on it before next year (🥁), but I can certainly pick it up then if needed.

neutrinoceros avatar Dec 19 '24 09:12 neutrinoceros

rebasing after #305 was just merged

neutrinoceros avatar Dec 23 '24 12:12 neutrinoceros

Instead of having a CI to check that the documentation builds, followed by RTD building it on their side, you could use a CI that builds the documentation and then pushes the HTML pages to GitHub Pages (if the build succeeds).

Here is a GitHub Action that I wrote for another project. Feel free to use it if you find it helpful.

smaret avatar Dec 23 '24 16:12 smaret

Thanks @smaret . For the moment nobody has time to migrate the documentation to Github Pages, so we will keep this as is.

glesur avatar Feb 21 '25 16:02 glesur