conda parallel run error

[julesghub]

Using conda in parallel models have this error along with the usual PETSC ERROR msg. ERROR: SCOTCH_dgraphInit: Scotch compiled with SCOTCH_PTHREAD and program not launched with MPI_THREAD_MULTIPLE

The work around is to put from mpi4py import MPI

before the import petsc4py or import underworld in your input file.

[julesghub]

The appropriate solution is likely to from mpi4py import MPI before everything in underworld3/__init__.py. Need to test that.

[knepley]

Here is the explanation. MPI code cannot be used with threads unless you promise to somehow coordinate. MPI_THREAD_MULTIPLE is one such promise, and it seems like Scotch requires this if you use threads. This is an option to MPI_Init(). Before calling PetscInitialize(), you can set PETSC_MPI_THREAD_REQUIRED = MPI_THREAD_MULTIPLE, to make this happen, or you can call MPI_Init() yourself, which is the solution above.

[julesghub]

Thanks for the info @knepley! I'll stick to the above solution for now and see how it goes.

[lmoresi]

@julesghub - I may have broken this issue again in development as I was trying to get back the ability to read command line arguments. We might need to check !

[julesghub]

roger that, I'll investigate today.

[lmoresi]

I still had trouble with this recently - can we revisit when the petsc 3.21.0 workflow is in place, @julesghub ?

[lmoresi]

By default, in our code, this is triggered from the conda-forge petsc. This and the ordering that allows command line arguments to be read seem to be a merry-go-round of cyclic regression.