dflow
dflow copied to clipboard
deepmodeling
Dflow is a Python framework for constructing scientific computing workflows (e.g. concurrent learning workflows) employing Argo Workflows as the workflow engine.
## Checklist * [ ] Checked the syntax is legal within dflow. * [ ] Tested using the latest version. ## Summary What happened/what you expected to happen? What version...
## dflow-helloworld.ipynb > from https://github.com/zjgemi/dflow_helloworld/blob/master/dflow-helloworld.ipynb ~~~python #Define a Python OP from pathlib import Path from dflow.python import OP, Artifact from dflow import SlurmRemoteExecutor slurm_remote_executor = SlurmRemoteExecutor( host= "xxx.xx.xxx.xx", port= xxx,...
- Ensure identical behaviors with the default mode for commonly used functions. - Supplement more UTs for the debug mode. - Improve the observability of the debug mode. E.g. display...
Current Bohrium executor runs an actual Pod with a go process or a DPDispatcher script running in the container to poll the job status. When massive steps with Bohrium executor...
Dflow is designed for scientific computing and has different focus in comparison to the general-purpose Argo. An independent frontend UI is necessary for dflow. The possible features include 1. Display...
At present, it is a bit complex for users to debug a step in a workflow with Argo. If we can reproduce the runtime environment in the container of a...
If a whole workflow runs on an HPC cluster. There is no need to upload/download files between the HPC fs and the artifact repository (e.g. S3). Of course one can...
There are some todos for wlm-operator, including but not limited to - Develop a robust agent for forwarding the red-box socket. It may retry under network interruptions. - Make configurator...
Synchronization of logs has not been implemented in dispatcher. For minimal modification of dispatcher, the dispatcher executor in dflow may synchronize logs via a sidecar container. Some suggestions 1. Run...
https://github.com/deepmodeling/dflow/issues/61 For some users, the current approach for constructing a loop (via a recursion of steps templates) may be kind of complex. May dflow provide a syntactic sugar for that?
Metadata
Owner
Metadata
Dflow is a Python framework for constructing scientific computing workflows (e.g. concurrent learning workflows) employing Argo Workflows as the workflow engine.