Add missing options to dask-ssh cli
Closes #3763
This PR add missing options from dask-scheduler and dask-worker commands to dask-ssh.
Example:
$ dask-ssh --nprocs 2 --nthreads 1 --worker-port 9000:9010 --hostfile ../nodes \ --scheduler-dashboard-address :8989 --scheduler-preload /path/to/foo.py
Thanks @jcrist for your feedback.
test_bokeh_deprecation keeps failing in CI checks. It passes on my local setup. Any idea why?
test_bokeh_deprecation keeps failing in CI checks. It passes on my local setup. Any idea why?
Not sure, but since you've changed the CLI here I'd expect this to be due to your changes. Usually when I run into a test failing I do the following:
- I find the name of the test, and use
py.testlocally run just that test:
# The `-k` flag tells pytest to run just that test
$ py.test distributed/cli/tests/test_dask_worker.py -k test_bokeh_deprecation
- If looking at the failure doesn't help, I might also run py.test with
--pdb, to drop into a debugger on failure
# The `--pdb` flag tells pytest to start a pdb session on error
$ py.test distributed/cli/tests/test_dask_worker.py -k test_bokeh_deprecation --pdb
- If that still isn't useful, I might make a separate script (or in this case CLI command) to reproduce and iterate from there. If needed, I might add a
breakpoint()call somewhere in the code where I want to stop and inspect what's going on. In this case, it looks like the--bokeh-portand--bokehflags are possibly no longer setting thebokeh_portandbokehparameters properly. You might start looking here: https://github.com/dask/distributed/blob/master/distributed/cli/dask_worker.py#L269-L279 aren't being forwarded properly.
If after looking through this you're still having trouble debugging, let me know and I'll take a closer look.
@jcrist it seems that it was an import order issue for Click options. pytest runs test_dask_ssh before test_dask_worker and will import and register Click Options for distributed.cli.dask_ssh.main before distributed.cli.dask_worker.main.
I modified some cli tests to run them using subprocess.Popen instead of CliRunner.invoke to avoid this import issue.
@jcrist please let me know if anything else is needed here. Thanks,