dask-drmaa
dask-drmaa copied to clipboard
dask
Deploy Dask on DRMAA clusters
There hasn't been any activity here for a couple of year. Is this project still used? Has it been replaced with something like dask-jobqueue? Should we archive it?
On `cluster.close()`, this error is thrown on current master: ``` Traceback (most recent call last): File "/home/smmanoet/.local/lib/python3.6/site-packages/tornado/gen.py", line 326, in wrapper yielded = next(result) File "/home/smmanoet/.local/lib/python3.6/site-packages/dask_drmaa/core.py", line 285, in stop_workers...
Noticed that `test_stop_single_worker` has started failing on CI. Seems to be a consistent failure. However it wasn't failing on the same code a month ago. So something else has changed...
In cases where there are lots of workers `stop_workers` is slow. This appears to be caused by [closing each worker in a `for`-loop]( https://github.com/dask/dask-drmaa/blob/0.2.1/dask_drmaa/core.py#L291 ). As the common case where...
Fixes https://github.com/dask/dask-drmaa/issues/65 Replaces https://github.com/dask/dask-drmaa/pull/81 Rewrites `DRMAACluster.scale_down` to use DRMAA to terminate the workers. Also makes use of `DRMAACluster.scale_down` in `DRMAACluster.stop_workers` (where this code was pulled from). With this change, it...
As discussed in PR #74, this PR allows user to specify a basename for the output and error path instead. The basename is then automatically completed to include the job...
Currently [our `_retire_workers` implementation]( https://github.com/dask/dask-drmaa/blob/ed71052280689dd6dc786b425f77c75c00db5b36/dask_drmaa/adaptive.py#L98-L113 ) implementation deviates from [Distributed's `_retire_workers` implementation]( https://github.com/dask/distributed/blob/1.21.4/distributed/deploy/adaptive.py#L208-L224 ). Would be nice if we could bring them back into correspondence and/or eliminate our copy of...
This PR makes initializing the DRMAACluster object with a user-defined script possible. Trivial fix, the script attribute was not set before using it when intializing the template.
I'm able trouble reproducing the unit tests based on the given instructions, for Windows. I've done the following: - Replaced all carriage returns in all `*.sh` files in the top-level...
