Rebuilding module breaks for HMNS if there are sort-of circular builddependencies (e.g. XZ and gettext)
I wanted to rebuild the module XZ-5.2.5-GCCcore-9.3.0.eb, it uses gettext-0.20.1.eb (from system level) as a builddep to resolve the circular dependency, but there is also a gettext-0.20.1-GCCcore-9.3.0.eb (which depends on XZ-5.2.5-GCCcore-9.3.0.eb)
So, what it seems like it's happening is
- Easybuild starts off with moving away the XZ module to a .bak file
- It preps the new one, but, as, some sanity check step along the way, it verifies that it can load all the dependencies and builddependencies of XZ,
- It first loads the
Core/GCCcore/9.3.0which expands the modulepath to now include the temporarily brokengettext/0.20.1,gettext-0.20.1-GCCcore-9.3.0.eb - It then tries to run
/usr/share/lmod/lmod/libexec/lmod python load gettext/0.20.1which was supposed to be the Core version, but, Lmod doesn't know that, so it finds the the temporarily broken one and spits out
== 2021-06-02 10:46:48,648 easyblock.py:3300 INFO Running method prepare_step part of step prepare
== 2021-06-02 10:46:49,031 modules.py:606 INFO Checking whether Core/gettext/0.20.1 exists...
== 2021-06-02 10:46:49,032 modules.py:611 INFO Module Core/gettext/0.20.1 exists (found in list of available modules)
== 2021-06-02 10:46:49,032 modules.py:634 INFO Result for existence check of Core/gettext/0.20.1 module: True
== 2021-06-02 10:46:49,032 modules.py:606 INFO Checking whether Compiler/GCCcore/9.3.0/binutils/2.34 exists...
== 2021-06-02 10:46:49,032 modules.py:611 INFO Module Compiler/GCCcore/9.3.0/binutils/2.34 exists (found in list of available modules)
== 2021-06-02 10:46:49,032 modules.py:634 INFO Result for existence check of Compiler/GCCcore/9.3.0/binutils/2.34 module: True
== 2021-06-02 10:46:49,032 modules.py:606 INFO Checking whether Core/GCCcore/9.3.0 exists...
== 2021-06-02 10:46:49,032 modules.py:614 INFO Module Core/GCCcore/9.3.0 not found in list of available modules, checking via 'module show'...
== 2021-06-02 10:46:49,379 modules.py:634 INFO Result for existence check of Core/GCCcore/9.3.0 module: True
== 2021-06-02 10:46:51,865 build_log.py:169 ERROR EasyBuild crashed with an error (at easybuild/base/exceptions.py:124 in __init__): Module command '/usr/share/lmod/lmod/libexec/lmod python load gettext/0.20.1' failed with exit code 1; stderr: Lmod has detected the following error: These module(s) or extension(s) exist but cannot be loaded as requested: "XZ/5.2.5"
Try: "module spider XZ/5.2.5" to see how to load the module(s).
which is expected, since the new XZ module isn't installed yet.
So, only affects HMNS of course because of the ambiguity of gettext/0.20.1
As a test, one can also just run with --module-only --installpath-modules=/tmp/empty_directory and it will work (since the old XZ module is still present, which means both gettext modules work).
Not sure how to address this. Problably just avoid loading in builddeps, they aren't relevant for the final module anyway, but I'm not sure what is triggering the loading of gettext/0.20.1 here. Something in prepare_step? I'm not sure what this does.