pecan icon indicating copy to clipboard operation
pecan copied to clipboard

The message will not be printed if we run the `write.config` function inside the `future_map`.

Open DongchenZ opened this issue 6 months ago • 3 comments

https://github.com/PecanProject/pecan/blob/edf1ece5237644078322f5daabe81c431171fcde/modules/uncertainty/R/ensemble.R#L233

DongchenZ avatar Jul 22 '25 15:07 DongchenZ

See https://github.com/PecanProject/pecan/blob/edf1ece5237644078322f5daabe81c431171fcde/modules/assim.sequential/R/sda.enkf_MultiSite.R#L446

DongchenZ avatar Jul 22 '25 15:07 DongchenZ

I was debugging the SDA workflow, and at the beginning, I couldn't find where it went wrong until I dove into the write.ensemble.configs function directly, which is very time-consuming. I think it will be nice to print out the message.

DongchenZ avatar Jul 22 '25 15:07 DongchenZ

I've noticed this as well -- not just this line, but all logger messages.

The standard Stack Overflow answer to questions like this is that collecting and displaying output from multiple (potentially remote) workers is more complicated than printing from a one-core serial session, but even knowing that I have trouble predicting when furrr will display output. I think it may wait for output from all workers and then reassemble it to display the lines in order?

For instance on my machine with future::plan("multisession", workers=5), future_map_int(1:30, \(x){message(x); if (x > 10) Sys.sleep(1); x}) displays messages 1 through 10 right away, then sits for about ten seconds, then displays 11-30 all at once.

I keep thinking furrr_options(stdout = TRUE) will help (it's documented as "standard output of the underlying futures is relayed as soon as possible"), but (1) it's the default already, and (2) the other choice of stdout = FALSE doesn't relay output slower, it discards it entirely.

infotroph avatar Jul 22 '25 18:07 infotroph