Include script directory to the search path?

[jrapin]

Feedback received:

It would be great if submitit also includes the script directory into its search path.
So the script could be running like non-submitit cases.

submitit is expected to work is if it were run locally, so adding the script path can definitely make sense if it has no border effect (anything to fear?)

[gwenzek]

I'm a bit mitigated. For me it seems that it only helps with code which is not in modules. How much should we bend backward for this use case ?

[gwenzek]

I have changed my mind on this.

The problem is that when running the python schedule_exp.py python will automatically add the current folder to sys.path. When the job starts it's started with python -m submitit and doesn't add "." which leads to confusing behavior.

[gwenzek]

Alternative implementation of #1596:

process = subprocess.Popen(
        [sys.executable, "-m", "submitit.local._local", str(folder)],
        shell=False,
        env=env,
        cwd=sys.path[0],  # <--- set the cwd to be the same than the one used for launching the script. 
    )

this need to be adapted for the SLURM executor. And it may lead to other errors if you have a custom manipulation of sys.path (eg that prepend something to the path and changes sys.path[0])

I think #1596 is more robust