Make submission script more human readable

[lesteve]

from dask_jobqueue import SLURMCluster 
cluster = SLURMCluster(cores=1, memory='1GB') 
print(cluster.job_script()) 

#!/usr/bin/env bash

#SBATCH -J dask-worker
#SBATCH -n 1
#SBATCH --cpus-per-task=1
#SBATCH --mem=954M
#SBATCH -t 00:30:00

/home/lesteve/miniconda3/bin/python -m distributed.cli.dask_worker tcp://192.168.0.11:44065 --nthreads 1 --memory-limit 1000.00MB --name name --nanny --death-timeout 60

Wouldn't it be nice if the last line use backslashes to be more human-readable i.e. something like:

/home/lesteve/miniconda3/bin/python -m distributed.cli.dask_worker \
    tcp://192.168.0.11:44065 \
    --nthreads 1 \
    --memory-limit 1000.00MB \
    --name name \
    --nanny --death-timeout 60

Full disclosure: not sure if there is an easy way to do this or if you have to write a little bit of code.

[mrocklin]

+1 from me if it's doable. This would be great.

On Tue, May 19, 2020 at 2:08 AM Loïc Estève [email protected] wrote:

from dask_jobqueue import SLURMCluster cluster = SLURMCluster(cores=1, memory='1GB') print(cluster.job_script())

#!/usr/bin/env bash

#SBATCH -J dask-worker #SBATCH -n 1 #SBATCH --cpus-per-task=1 #SBATCH --mem=954M #SBATCH -t 00:30:00

/home/lesteve/miniconda3/bin/python -m distributed.cli.dask_worker tcp://192.168.0.11:44065 --nthreads 1 --memory-limit 1000.00MB --name name --nanny --death-timeout 60

Wouldn't it be nice if the last line use backslashes to be more human-readable i.e. something like:

/home/lesteve/miniconda3/bin/python -m distributed.cli.dask_worker
tcp://192.168.0.11:44065
--nthreads 1
--memory-limit 1000.00MB
--name name
--nanny --death-timeout 60

Full disclosure: not sure if there is an easy way to do this or if you have to write a little bit of code.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/dask/dask-jobqueue/issues/431, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTFUVXN3Y5JOTXIL4J3RSJEA5ANCNFSM4NE2BNJA .

[guillaumeeb]

This looks pretty doable! At first I thought we should wait for #370, but I'm not sure.

[AdamWlodarczyk]

Hi, I've made some code tweaks for this but since #370 is back to life it is not relevant I think. Or is it?

[guillaumeeb]

You're right @AdamWlodarczyk, if we managed to finish #370, it may be used to solve this one, although I'm not completely sure of it... Maybe you can propose a PR if you've got something ready?

[AdamWlodarczyk]

@guillaumeeb OK, I proposed a PR. htcondor and sge have some problems but I don't think they are related to this PR. I tried those with master branch with same result.

[guillaumeeb]

I'm going to close this issue as it seems it is only a nice to have feature which would involved too many changes in the code.

But if anyone has a smart idea to implement it with few change, I'm fully OK to reopen!