dftlibs
How to specify number of cores for parallel grid generation?
In the readme it is mentioned that
The Becke partitioning step is parallelized using Rayon. In other words, this step should be able to use all available cores on the computer or computing node. Since grids are currently generated atom by atom, it is also possible to parallelize "outside" by the caller.
Indeed when I run the code, the program has 1600% CPU usage on my 8 core machine, indicating that it is trying to use all the threads possible.
Such a behavior is not very desirable as first of all, parallelizing over 16 threads on an 8 core machine is not very efficient as in my experience. Furthermore, this very problematic when submitting jobs as you need to specify a number of cores and this leads to oversubscription.
Well actually, I found the solution as soon as I posted. I just googled about how to limit the number of threads used by RAYON, and the official FAQ (https://github.com/rayon-rs/rayon/blob/master/FAQ.md) has the solution.
If you want to alter the number of threads spawned, you can set the environmental variable RAYON_NUM_THREADS to the desired number of threads or use the ThreadPoolBuilder::build_global function method
So I just needed to do
export RAYON_NUM_THREADS=4
Thanks for reporting! I am reopening because I would like to add your solution to the README and will close as soon as I do that.
