Errors when running prophet inside mclapply

[jonlachmann]

When trying to run prophet inside mclapply I get an error about a timeout as follows

function ()  : Timeout exceeded

I have created a minimal example to demonstrate the problem. First, four models are created inside mclapply, causing the errors. Then a demonstration of a successful run in sequence is shown. This also seems to have the effect to make running further models in parallel work afterwards. Finally we run four models inside mclapply which then works.

My guess is that the stan model is not compiled before the first run, and this is for some reason blocking parallel runs. Would it be possible to have this work without having to run a non parallel run beforehand?

Please let me know if any further information is needed, or if I can be of any further assistance.

library(prophet)
library(parallel)

# Create some dummy data
test_df <- data.frame(ds=as.Date(paste0("2000-", 1:12, "-01")), y=1:12)

# Create 4 models in parallel
models_par <- mclapply(seq_len(4), function (x) prophet::prophet(df=test_df), mc.cores=4)

# Create 4 models sequentially
models_seq <- vector("list", 4)
for (i in seq_len(4)) {
  models_seq[[i]] <- prophet::prophet(df=test_df)
}

# Create 4 models in parallel after running them sequentially
models_par2 <- mclapply(seq_len(4), function (x) prophet::prophet(df=test_df), mc.cores=4)

sum(sapply(models_par, class) == "try-error") # 4
sum(sapply(models_seq, class) == "try-error") # 0
sum(sapply(models_par2, class) == "try-error") # 0

[tcuongd]

Hmm if you're using the default rstan backend, the model should already be compiled as a part of installing the package. I'm not sure if there's an issue with calling C++ routines inside mclapply.