[BUG] Nodes fail to run in interactive sessions when using custom output names
Describe the bug
Using the file_prefix argument in node.validate , then attempting to run the node within an interactive session with BSS.Node.run causes the following exeption: FileNotFoundError: [Errno 2] No such file or directory: 'output.yaml'.
This is due to the assumption here that the output file will always be called output.yaml, even if file_prefix is set to something different.
To Reproduce Steps to reproduce the behavior:
- Create a basic node e.g.:
import BioSimSpace as BSS
node = BSS.Gateway.Node("A node.")
node.addInput("in", BSS.Gateway.Integer(help="in"))
node.addOutput("out", BSS.Gateway.Integer(help="out"))
node.setOutput("out", 1)
node.validate(file_prefix="test")
- Attempt to run the node in an interactive session:
In [1]: import BioSimSpace as BSS
INFO:numexpr.utils:Note: NumExpr detected 20 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 16.
INFO:numexpr.utils:NumExpr defaulting to 16 threads.
In [2]: BSS.Node.setNodeDirectory(".")
In [3]: BSS.Node.run("null_node", {"in":1})
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <module>:1 │
│ │
│ /home/matt/code/biosimspace/python/BioSimSpace/Node/_node.py:160 in run │
│ │
│ 157 │ │
│ 158 │ if proc.returncode == 0: │
│ 159 │ │ # Read the output YAML file into a dictionary. │
│ ❱ 160 │ │ with open("output.yaml", "r") as file: │
│ 161 │ │ │ output = _yaml.safe_load(file) │
│ 162 │ │ │
│ 163 │ │ # Delete the redundant YAML files. │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
FileNotFoundError: [Errno 2] No such file or directory: 'output.yaml'
(please complete the following information):
- OS: Ubuntu 22.04.5 LTS
- Version of Python: 3.11
- Version of BioSimSpace: latest
devrelease - I confirm that I have checked this bug still exists in the latest released version of BioSimSpace: [yes]
I've been trying to run multiple instances of a node simultaneously using dask, and I now see that this issue may be more difficult to solve than I initially thought.
In order for these to work robustly in parallel we need to be able to generate unique filenames for both input and output yaml files. This is easy enough with inputs, but with outputs it is more complex since the unique filename will likely need to be generated at runtime, meaning that the name isn't known until the node is run.
The easiest thing I can think of would be to use an input to define the name of the output.