WISH: parallelly::availableCores() instead of parallel::detectCores()
(disclaim: I'm the author) To minimize the risk of overusing CPUs on multi-tenant and multi-process machines, but also to work in more environments, please consider changing:
https://github.com/microsoft/finnts/blob/57207bf18b79949244017c4df8ec1296a9da0752/R/general_parallel.R#L6-L13
to
get_cores <-function(num_cores){
min(c(parallelly::availableCores(omit = 1), num_cores))
}
This also has the benefit of working on single-core hosts, where detectCores() == 1, e.g. free RStudio Cloud accounts, some k8s/container setups, etc. For more arguments, see https://parallelly.futureverse.org/#availablecores-vs-paralleldetectcores.
Hey Henrik, thanks for reaching out. I'll look more into this and incorporate into package soon. Our main focus right now is to build out new parallel compute options like spark.