dstack
dstack copied to clipboard
doc: clarify behavior when running workflow without tag on dependency
The following behavior of workflow run needs to be illustrated in the doc.
For a workflow file (note the lack of tag :latest in the dependency:
workflows:
- name: download
provider: python
requirements: requirements.txt
script: mnist/prepare_data.py
artifacts:
- data
- name: train
provider: python
requirements: requirements.txt
script: mnist/train.py
depends-on:
- download
artifacts:
- model
resources:
gpu: 1
running the sequence of commands:
dstack run download
# wait for completion, then run
dstack run train
will yield this:
the first command creates a run
warm-implala-1. The second command doesn't usewarm-implala-1, instead it runs from the start.
expectation/improvement
- first expectation was
trainwould use whichever the latestdownloadwas, regardless of tag being specified. - the user flow to: run
download-> wait for completion -> taglatestin UI -> come back and runtrainis quick a lot of manual steps and friction. - suggestion:
when no tag is specified in dependency, do run from scratch, that's fine. But when alatesttag is specified, automatically use the latest one. This means thatlatesthas to be a reserved tag that always points to the latest run of a workflow - this is also semantically consistent. If user wants to manually use special tag likefoo_1then specify it, they can follow the usual tag usage as the doc currently explains.
