DRAGONS icon indicating copy to clipboard operation
DRAGONS copied to clipboard

Multiprocessing issues with "differential evolution"?

Open chris-simpson opened this issue 2 years ago • 0 comments

The use of "differential evolution" as the default optimizer in KDTreeFitter appears to have led to some issues due to its use of multithreading. These include:

  1. Large numbers of open "files" requiring ulimit -n 1024
  2. Inability to run python scripts that do not have the if __name__ == "__main__" idiom
  3. Issues running scripts in an interactive python environment (probably related to the previous item)

The second is well-understood here but may trip up people who don't code to a high standard. The other two are less clear, and it's also unclear whether any of this is due to the change from "fork" for "spawn" for python multiprocesses on OSX in python 3.8+.

Some more investigation/reports might be needed, as well as possibly exposing a parameter to use a different method of optimization.

chris-simpson avatar Apr 28 '23 00:04 chris-simpson