bionode-watermill
bionode-watermill copied to clipboard
bionode
💧Bionode-Watermill: A (Not Yet Streaming) Workflow Engine
"External" inputs being referred to here as those which come from the canonical `inputs.json` (though it can be named anything). See the [CWL Generic Execution Process](https://www.commonwl.org/v1.0/CommandLineTool.html#Generic_execution_process) 1. Load, process and...
Due to the way we [parse a string into a child process](https://github.com/bionode/bionode-watermill/blob/46499966a41c5977941ba4b617298aa64192b2c3/lib/utils/shell.js#L12) it may be the case that certain operations (e.g., a bash for/while read loop) may not run. It...
In order to be a *streaming* workflow engine, we need to support the ability to `pipe` between tasks. This does not add much to the "traditional" pipeline which mostly reads/writes...
Todos: - [ ] keep stdout/stderr logs in main logs? - [ ] set absolute path to `stdout.txt` and `stderr.txt` in task state - [ ] how to handle this...
Files could have validators to check if it is: - [ ] correct file size - [ ] correct file extension - [ ] support custom validators #42
See https://github.com/bionode/bionode/issues/34
This means to be able to pass a custom file validator on a per task basis, something like: ```javascript const mytask = task({ input: '*.sra', output: '*.vcf' validators: (file) =>...
I am not sure what may be causing this issue, but I have made several attempts on running a pipeline after another has been executed. You can check them [here](https://github.com/bionode/GSoC17/blob/master/Journal/Week_12.md#execution-of-pipelines-after-another)...
While doing my [GSoC17 Week 12](https://github.com/bionode/GSoC17/blob/master/Journal/Week_12.md#through-tasks) report noticed that through tasks properly handle input and output but it does not have `resolvedInput` properly logged to `graphson.json` and there is no...
Current implementation on task definition by the user is somehow javascript dependent. Example: ```javascript const gunzipIt = task({ input: '*_genomic.fna.gz', output: '*.fa', params: { output: 'uncompressed.fa' } }, ({ params,...