chakravala
Base.Meta.ParseError("invalid operator \"--\"") when multiprocessing
Hello,
I'm using your package with a huge list of expression. Basically, I'm doing this:
res = map(exp -> (exp, rcall(exp)), res)
It run correctly but it is taking forever and my CPU is barely used.
So I wanted to speed things up with multiprocessing. I tried with pmap, @threads, @spawn and @distributed and starting with julia -p 6 in deed.
Most of the time I get this error :
ERROR: LoadError: On worker 2:
Base.Meta.ParseError("invalid operator \"--\"")
Or no error but no increase in speed (and CPU < 10%).
Moreover, I have this error without doing any multiprocessing.
With the same code I can crash if I start julia ./script.jl
but I don't crash inside the REPL julia> include("GenerateFormula4.jl")
I'm using Windows 10.
I'm still new to Julia so I'm not sure what could cause the problem.
The Reduce.jl package is not currently designed to handle multiprocessing, since communication is based on a single pipe communicating with another process. I've thought about specifically adding some feature to open multiple separate callable instances of the reduce process, but never tried it yet.
Thank you for your prompt reply. I'll be glad to try this new feature. Meanwhile I will try to find a workaround.
Reduce.jl is fairly quick and responsive, but is limited in performance by the character stream interface. It's possible to create a faster interface for Reduce, but it would not be a simple task.
Note that you can also send multiple commands to Reduce at once (like an expression block).