error: static_assert failed due to requirement 'sizeof(long) == sizeof(unsigned int)' "ParMETIS and libMesh ID sizes must match!"

[yurivict]

libmesh-1.6.2.log

OS: FreeBSD 13

[jwpeterson]

I think the issue is caused by mixing system Metis:

--with-metis=/usr/local

with the Parmetis that comes with libmesh, since Parmetis inherits idx_t from whatever metis you are using.

In the metis.h distributed with libmesh, we have this code to set the IDXTYPEWIDTH consistently with LIBMESH_DOF_ID_BYTES:

#if LIBMESH_DOF_ID_BYTES == 8
#  define IDXTYPEWIDTH 64
#else
#  define IDXTYPEWIDTH 32
#endif

followed by

#if IDXTYPEWIDTH == 32
  typedef int32_t idx_t;
  ...
#elif IDXTYPEWIDTH == 64
  typedef int64_t idx_t;
  ...
#endif

So my guess is that your system metis.h sets IDXTYPEWIDTH to 64, but you are using a libmesh with 32-bit dof_id_type, hence the mismatch. I see that you passed --disable-strict-lgpl to configure, so this is what enables parmetis, otherwise it is disabled by default.

[yurivict]

Not supplying --with-metis=/usr/local solves the compilation problem, but now it installs include/metis.h which conflicts with the externally installed metis.

[yurivict]

metis.h isn't included from any other headers that libmesh installs so it looks like a bug that it is installed.

[jwpeterson]

Hmm... although none of the libmesh headers include metis.h directly, we have include/partitioning/parmetis_helper.h, which includes parmetis.h, and that in turn includes metis.h. I can't quite remember exactly what the point of the ParmetisHelper helper is, but the docs say it's intended to be a pointer-to-implementation class, so it's unfortunate that it breaks encapsulation like this. Maybe it's possible to fix by moving the ParmetisHelper class to parmetis_partitioner.C?

[jwpeterson]

Oh, I see now that parmetis_helper.h is not actually installed, so that's good. So yes, it seems to me that we should not install metis.h.

[roystgnr]

It's actually quite on purpose that we install metis.h when we're configured to use our own copy. Mostly that's so libMesh user code can assume it's safe to make direct Metis API calls, partly that's because it's often a good thing when conflicts between Metis versions crop up earlier rather than later. Overwriting another's header file is embarrassing, but it's usually easier to root out than the ABI conflicts that can come up when an application tries to use one header file but then tries to link to a library with an incompatible set of symbols, or worse if linking succeeds but there's an ABI incompatibility that hits at runtime.

I say "usually", though, but that doesn't seem to apply here, does it? I don't actually see a fix for this other than "install an external parmetis that's compatible with the external metis" or "configure --disable-parmetis". Which is kind of annoying; without Parmetis we can only repartition a distributed mesh by temporarily serializing it, making it entirely impractical to do load-rebalancing for DistributedMesh transient adaptive refinement.

[yurivict]

@roystgnr

Now with libmesh-1.7.0 this failure occurs with --with-metis=/usr/local as well. The port uses external metis.

This condition should be checked during the configure stage.

Without --with-metis=/usr/local another error occurs:

In file included from src/partitioning/parmetis_partitioner.C:33:
./include/libmesh/parmetis_helper.h:38:15: fatal error: 'parmetis.h' file not found
#     include "parmetis.h"
              ^~~~~~~~~~~~

[jwpeterson]

@yurivict if I understand correctly, there is no parmetis.h installed in /usr/local on your system? So I think what should happen is that configure should detect this automatically and disable parmetis in the build. I checked briefly and I don't think there have been any updates, so it looks like the same bug is probably in master as well.

[yurivict]

if I understand correctly, there is no parmetis.h installed in /usr/local on your system?

Yes, there's no parmetis.h.