A for loop inside the objective function

[zxjroger]

Hi Tamás, Thank you for your package. I have a quick question. My objective function has a nested parallel for loop, something like

function objective(parameter)
    temp = Matrix(undef, row, col)
    np = nprocs()
        for t = 1:T
            i = 1
            nextidx() = (idx=i; i+=1; idx)
            @sync begin
                for p = 1:np
                    if p != myid() || np == 1
                        @async begin
                            while true
                                idx = nextidx()
                                if idx > num_simulation
                                    break
                                end
                                temp[t,idx] = remotecall_fetch(func, p, argument[t,idx])
                            end
                        end
                    end
                end
            end
        end
    end
    return scaler_output_fn(temp)
end

So, it uses remotecall_fetch to parallel inside the objective function.

In your package, https://github.com/tpapp/MultistartOptimization.jl/blob/88a326f3c9649d89ac1f504c5cab3fc2494c7c08/src/MultistartOptimization.jl#L63 The function sobol_starting_points() calls @spawn to compute the value of objective function at different points in parallel. So I have a parallel for loop nested in a parallel for loop. I have the impression that such "for loop" may be dangerous as discussed here. Do you think this may cause an issue?

Thank you.

[zxjroger]

I believe the code got stuck at some point when evaluate the objective function. It's either no data get fetched from other process, or no data get transmitted to other process. I didn't get any errors.

[tpapp]

I am not an expert on multithreaded computations, but I am happy to fix this if you have a suggestion. At this point I am not even sure what the issue is, or how the thread you linked applies. Also, an MWE I can run would help.

[zxjroger]

Thanks for your response. For now, I just change the code in function sobol_starting_points() from

map(fetch, map(x -> @spawn(LocationValue(x, objective(x))), Iterators.take(s, N)))

to

map(x -> LocationValue(x, objective(x)), Iterators.take(s, N))

so it does not compute the value of objective function in parallel.

[zxjroger]

I have another question, is it possible to query exit flag?

[castrovjm]

Thanks for your response. For now, I just change the code in function sobol_starting_points() from

map(fetch, map(x -> @spawn(LocationValue(x, objective(x))), Iterators.take(s, N)))

to

map(x -> LocationValue(x, objective(x)), Iterators.take(s, N))

so it does not compute the value of objective function in parallel.

Hi,

I currently have the same problem as you do: parallelization inside the objective function. How do I do the change you are proposing? where can I find this script?

[castrovjm]

@tpapp do you think you can include the parallelization of the starting points as an option?

[zxjroger]

@castrovjm After you add this package in Julia, you should find it in the package folder. On Mac OS, it is "~/.julia/packages/MultistartOptimization". Then, look for the source code folder "src". You can find the Julia script "MultistartOptimization.jl". Edit this script as I proposed and save.

[tpapp]

I currently have no time to work on this, but I am still happy to take PRs.

[atrophiedbrain]

#10 Added parallel_sobol_points keyword parameter to multistart_minimization function to address this issue.