dask-yarn
dask-yarn copied to clipboard
Not working when the worker node has multiple IPs
If the worker node has multiple IPs, the worker process may bind to the wrong one. I use this workaround at https://github.com/dask/dask-yarn/blob/d576f11dd7d32299bd105e665694e54fc1c7055e/dask_yarn/cli.py#L463
I add to the Nanny() arguments, so that it looks like:
worker = Nanny(
scheduler,
loop=loop,
memory_limit=memory_limit,
worker_port=0,
nthreads=nthreads,
name=skein.properties.container_id,
listen_address="0.0.0.0",
host="0.0.0.0",
)