modules icon indicating copy to clipboard operation
modules copied to clipboard

Make Python init more language native

Open daytonb opened this issue 3 months ago • 3 comments

This pull request is an effort to implement a more language-native way to get the "module" function in Python scripts. It is the Python portion of #483. I first proposed this pull request in the mailing list. Below is a description of the plan from that conversation.

For clarity I'll use "Python-module" to refer to a Python script defining functions/classes that you import in Python code and "environment-module" to refer to a TCL environment module that you module load/module unload/etc.

The current documentation for Python initialization (https://modules.readthedocs.io/en/latest/module.html#examples-of-initialization) has

import os
exec(open("/usr/share/Modules/init/python.py").read(), globals())
module("load", "modulefile", "modulefile", "...")

A more Pythonic approach would be to import this module function somehow. Currently, you can do that with as follows

import sys
sys.path.append("/usr/share/Modules/init")
from python import module
module("load", "modulefile", "moduelfile", "...")

The issues with this are that you have to modify the sys.path inside the script and since the Python-module import-ed is called "python". The name makes it look like module is a function from some core Python-module from the Python project.

In this pull request I will

  1. rename the Python initialization file to env_module.py (this is what lmod named theirs by the way) and
  2. to have the shell initialization in tcl/subcmd.tcl.in add the folder of initilization scripts to the PYTHONPATH environment variable.

These changes would change Python initialization to

from env_module import module
module("load", "modulefile", "moduelfile", "...")

which cleans up the way the Python script imports the module function and communicates more clearly that it comes from a 3rd-party Python-module related to environment modules.

daytonb avatar Nov 14 '25 16:11 daytonb

At this time there are a couple of issues I know I need to address:

  1. Add tests - I'm guessing we probably want some tests added
  2. Resolve test failures - I ran the quicktests locally and got 42 failures, but I couldn't see how you see which tests failed in order to figure out what I broke
  3. Have someone do a code a documentation review
  4. Change the python.py file to an install-time symlink on Linux:
    • In the mailing list there was discussion that for compatibility the installation process on Linux should symlink the new env_modules.py to python.py, and that no such compatibility in neccessary for Windows because the Windows installs have never had the python.py file.

daytonb avatar Nov 14 '25 16:11 daytonb

I've addressed the issues noted above except the failing tests. I still need to:

  • find and fix the failing tests
  • add new tests for the pythonpath configure options

daytonb avatar Nov 20 '25 05:11 daytonb

I pushed up some fixes. I want to rebase/squash them, but I need to sign off for now.

daytonb avatar Nov 22 '25 01:11 daytonb