renku-python
renku-python copied to clipboard
Exporting cwl and running it in project does not always produce the same output
Assume a user does the following:
renku run --name step1 -- echo \"some\nrandom\ncontent\" > input1
renku run --name step2 -- wc -l input1 > output1
renku workflow compose combined1 step1 step2
renku run --name step3 -- wc -l output1 > output2
renku run --name step4 -- cat output2 > output3
renku workflow compose combined2 step3 step4
renku workflow compose combined_all combined1 combined2
It is reasonable to expect that if they were to export the workflow as cwl, delete the output and then run the workflow then the workflow would recover the files exactly as they were. But that is not the case. Because the output depends on the context.
So you run the following commands:
renku workflow export --format cwl combined_all > test.cwl
rm output*
cwltool test.cwl
git diff output1
The diff for any output file is not empty. It has something like this:
diff --git a/output1 b/output1
index 4c3994d..a10b54e 100644
--- a/output1
+++ b/output1
@@ -1 +1 @@
-1 input1
+1 /tmp/g0ar9p7v/input1
Renku workflow execute provides the same output. We should adjust the context/behaviour of the cwl export/execution to avoid this.