dask-yarn
dask-yarn copied to clipboard
dask
Deploy dask on YARN clusters
I setup a new virtual environment and loaded Dask/Dask-Yarn and cannot import due to a deprecated package. I searched for Dask-Yarn and format_bytes and only found the deprecation warning in...
Hi, The [current method](https://yarn.dask.org/en/latest/environments.html#using-python-environments-local-to-each-node) for starting a conda environment local to each node does not work correctly: ```python from dask_yarn import YarnCluster # Use a conda environment at /path/to/my/conda/env cluster...
**What happened**: dask-yarn is not able to initialize the cluster, it runs indefinetly **What you expected to happen**: Cluster being initialized correctly **Minimal Complete Verifiable Example**: ```python from dask_yarn import...
**Describe the issue**: I used [this](https://github.com/dask/dask-yarn/blob/main/deployment_resources/aws-emr/bootstrap-dask) script to start EMR , but it take so long, I just used default option `-- jupyter` **Minimal Complete Verifiable Example**: ```python # Put...
Hi guys, I am sorry if it's a dummy/repeated question. We have been trying to follow the example to bring up dask on yarn and keep getting error "Kerberos ticket...
**What happened**: I tried to instantiate an EMR cluster with the AWS EMR bootstrap script. The provisioning of the cluster failed saying "bootstrap action 1 returned a non-zero return code"....
**What happened**: error while creating Client object **What you expected to happen**: correct init **Minimal Complete Verifiable Example**: ```python from dask_yarn import YarnCluster from dask.distributed import Client # Create a...
I'm trying to run a dask yarn_cluster job on dataproc but i can't. Dask worker log is giving me this issue: .skein.sh: line 2: environment/bin/python: No such file or directory...
From https://github.com/dask/dask-yarn/issues/49#issuecomment-456645866. I think this would look like: - A `worker_files` kwarg, which takes a mapping to forward to the `files` kwarg in the skein specification. - A `worker-file` and...
**What happened**: Connecting to a remote EMR cluster from a Jupyter notebook (using YarnCluster for Dask Cluster creation) causes notebook cell to hang. The YarnCluster client is able to successfully...
