Add default_slurm_ags option to SlurmEnvironment __init__

[metasoarous]

This is handy for submitting srun jobs through salloc, where the srun commands must all be run with --ntasks=1 for things to work properly. It's annoying to have to set this flag on every single Command.

[metasoarous]

Hmm... just realized maybe this should be srun_args instead of slurm_args since SAlloc also takes a slurm_args argument. Would reduce possible confusion.

[cmccoy]

I've been running a similar workflow, but in the shell rather than via scons. Setting the following environment variables prior to invoking srun works for me:

export SLURM_NTASKS=1
export SLURM_NPROCS=1
export SLURM_NNODES=1

For example, the script

#!/bin/sh
export SLURM_NTASKS=1
export SLURM_NPROCS=1
export SLURM_NNODES=1
srun hostname

Generates one line of output within an salloc with any -n argument.

Not sure if that would end up being easier here.

[metasoarous]

Admittedly, I prefer solutions that avoid messing with the action string, but I wasn't able to get the environment variables to work with scons. This slurm_args solution ended up being pretty easy.

On Fri, Feb 21, 2014 at 9:43 AM, Connor McCoy [email protected]:

I've been running a similar workflow, but in the shell rather than via scons. Setting the following environment variables prior to invoking srunworks for me:

export SLURM_NTASKS=1export SLURM_NPROCS=1export SLURM_NNODES=1

For example, the script

#!/bin/shexport SLURM_NTASKS=1export SLURM_NPROCS=1export SLURM_NNODES=1 srun hostname

Generates one line of output within an salloc with any -n argument.

Not sure if that would end up being easier here.

Reply to this email directly or view it on GitHubhttps://github.com/nhoffman/bioscons/issues/14#issuecomment-35754180 .

[nhoffman]

@metasoarous - would environment variables passed to the Environment constructor take care of this? For example, I typically set up the environment like this:

env = SlurmEnvironment(
    ENV = dict(
        os.environ,
        PATH=':'.join(['bin', path.join(venv, 'bin'), '/usr/local/bin', '/usr/bin', '/bin']),
        SLURM_ACCOUNT='fredricks_d'),
    variables = vars,
    use_cluster=use_cluster,
    shell='bash'
)

[metasoarous]

Yeah, I tried setting there, but wasn't able to get it to work.

On Fri, Feb 21, 2014 at 10:18 AM, Noah Hoffman [email protected]:

@metasoarous https://github.com/metasoarous - would environment variables passed to the Environment constructor take care of this? For example, I typically set up the environment like this:

env = SlurmEnvironment( ENV = dict( os.environ, PATH=':'.join(['bin', path.join(venv, 'bin'), '/usr/local/bin', '/usr/bin', '/bin']), SLURM_ACCOUNT='fredricks_d'), variables = vars, use_cluster=use_cluster, shell='bash' )

Reply to this email directly or view it on GitHubhttps://github.com/nhoffman/bioscons/issues/14#issuecomment-35757615 .

[cmccoy]

For me, this SConstruct creates a single file and prints a single hostname when run under salloc -n 3:

import os

from bioscons.slurm import SlurmEnvironment

env = os.environ.copy()
for k in ('SLURM_NTASKS', 'SLURM_NPROCS', 'SLURM_NNODES'):
    env[k] = '1'
env['SLURM_TIMELIMIT'] = '2:00'

e = SlurmEnvironment(ENV=env)

e.Command('fake', [], 'env | grep -P "SLURM|SALLOC|SRUN" > ~/env.$$SLURM_TASK_PID && hostname')

[metasoarous]

Ah... I think I see what I did wrong. I was looking at http://www.scons.org/doc/2.0.1/HTML/scons-user/x1741.html and trying to set env["ENV"]["SLURM_NTASKS"].

On Fri, Feb 21, 2014 at 11:30 AM, Connor McCoy [email protected]:

For me, this SConstruct creates a single file and prints a single hostname when run under salloc -n 3:

import os from bioscons.slurm import SlurmEnvironment env = os.environ.copy()for k in ('SLURM_NTASKS', 'SLURM_NPROCS', 'SLURM_NNODES'): env[k] = '1'env['SLURM_TIMELIMIT'] = '2:00' e = SlurmEnvironment(ENV=env) e.Command('fake', [], 'env | grep -P "SLURM|SALLOC|SRUN" > ~/env.$$SLURM_TASK_PID && hostname')

Reply to this email directly or view it on GitHubhttps://github.com/nhoffman/bioscons/issues/14#issuecomment-35764461 .

[cmccoy]

That looks right to me (if env is a SlurmEnvironment), but you need SLURM_NPROCS in addition to SLURM_NTASKS -- salloc sets both.

[metasoarous]

Huh... somehow it works with just srun --ntasks=1 though.

On Fri, Feb 21, 2014 at 11:42 AM, Connor McCoy [email protected]:

That looks right to me (if env is a SlurmEnvironment), but you need SLURM_NPROCS in addition to SLURM_NTASKS -- salloc sets both.

Reply to this email directly or view it on GitHubhttps://github.com/nhoffman/bioscons/issues/14#issuecomment-35765667 .