distributed
distributed copied to clipboard
dask
A distributed task scheduler for Dask
Server startup and shutdown is currently confusing and error prone see https://github.com/dask/distributed/pull/6615 and [`a8244bd` (#6603)](https://github.com/dask/distributed/pull/6603/commits/a8244bde99182ffb7abd6508285d566414f95b22) patterns evolving concurrent and re-entrant close and cancellation are prone to deadlocks: ```python try: self.comm.read()...
Looking through the whole codebase, this is the only place (not in tests) that a host of `127.0.0.1` is assigned: https://github.com/dask/distributed/blob/3551d1574c9cd72d60197cc84dd75702ebcfec54/distributed/comm/inproc.py#L43 Under certain circumstances, this OSError will throw and the...
``` _________________________________ test_simple __________________________________ args = (), kwds = {} @wraps(func) def inner(*args, **kwds): with self._recreate_cm(): > return func(*args, **kwds) ../../../miniconda3/envs/dask-distributed/lib/python3.8/contextlib.py:75: _ _ _ _ _ _ _ _ _...
I believe this issue is related to this [SO](https://stackoverflow.com/questions/56176567/dask-not-respecting-worker-directive-constraint-workers-allow-other-work) post. I'm finding that, despite resource restrictions, the dask-scheduler will assign keys to worker nodes even when I'm specifying otherwise. For...
Follow-up addressing https://github.com/dask/distributed/pull/6829#discussion_r939007236 - [ ] Tests added / passed - [x] Passes `pre-commit run --all-files`
We currently expose a number of functions and fixtures in `distributed.utils_test`. While I would consider these implementation details of our tests, and not part of any stable user-facing API, other...
Closes #6374 - [x] Tests added / passed - [x] Passes `pre-commit run --all-files` Adds an `absolute` kwarg to `Client.wait_for_workers` that makes it wait for exactly `n_workers` instead of the...
`BatchedSend` has a few edge cases which can silently put it into a broken state. (See my review on https://github.com/dask/distributed/pull/5457 for an enumeration of its issues.) Despite being a core...
This hasn't actually failed yet on the test report, but I know it could like it did here: https://github.com/dask/distributed/pull/6504#issuecomment-1150423010 This is caused by https://github.com/dask/distributed/issues/6390 in fallout from https://github.com/dask/distributed/pull/6361. ```python def...
This is an issue to track efforts towards some modernization efforts of our asyncio infrastructure. Very roughly this is about - Addressing all warnings triggered by py3.10 - Slowly migrating...
