submitit

submitit copied to clipboard

Hello! I am using slurm on 4-gpu servers using Submitit and I keep getting this unexpected srun error: ``` submitit WARNING (2022-08-12 02:06:11,453) - Caught signal SIGUSR2 on SERVER_NAME: this...

ofir1080

[Custom python environment]

11

Hi all ! In the context of cluster computing, it is sometime necessary to have the jobs running with a local python environment and not the one from the central...

emilemathieu

How to specify GPUs when executing locally?

4

I've successfully used `submitit` to submit jobs to our SLURM cluster, and overall the library works great. However I'm often faced with a situation where I need to work locally...

j0ma

Fallback to slurm for TorchDistributedEnv

1

jrapin

Submitit puts all tasks on a single GPU

3

Hi, I am trying to train on an AWS EC2 G5 node with eight A10G GPUs. I am running into a CUDA out-of-memory issue with an error message `Tried to...

Bai-YT

[Tentative] Rerun jobs more easily

1

jrapin

Hi, I'm trying to submit a huge job array to multiple partitions such as dev1, dev2.. and so on for example, ``` executor = submitit.AutoExecutor(folder='log') executor.update_parameters(slurm_partition="dev1,dev2,dev3", slurm_array_parallelism=50000) jobs = []...

MinkyuHa

Progress Bar for Jobs (Implementation)

3

Hi :) Thanks for creating this awesome open source repo, it helps me a lot! I wrote a function that tracks jobs` status with a progress bar. Perhaps it will...

yuvalkirstain

NodeList Declaration

1

Hi, Is it possible to declare the nodeList somehow (especially on slurm)?

Bontempogianpaolo1

Fix truncation of "Executor" in executor class name

4

Noticed this while trying to make my own plugin/Executor... `rstrip` can remove too much!

tmct