arpack-ng icon indicating copy to clipboard operation
arpack-ng copied to clipboard

Automatic linking with openblas

Open cpviolator opened this issue 6 years ago • 10 comments

I would like to build arpack-ng as a dependency to a higher level library. Arpack-ng will automatically link to an installed openblas shared library on some systems. This openblas library is not linked in the higher level library, causing link errors at build time with CMake.

How can I pass a CMake flag to arpack-ng to block this linking to openblas so that ARPACK builds it own blas libs?

cpviolator avatar May 26 '19 06:05 cpviolator

If hll (higher level lib) does not need to link to openblas, but, does need to link to arpack-ng (which links to openblas), then hll should not see openblas : all should be fine. Are you sure hll does not link to openblas : what does ldd says ? Compile / link traces ? Error message ?

How can I pass a CMake flag to arpack-ng to block this linking to openblas so that ARPACK builds it own blas libs?

Not sure arpack-ng builds it's own blas. I would say arpack-ng uses any blas it finds... And fails compiling if no blas found.

fghoussen avatar May 26 '19 14:05 fghoussen

At the moment, we download and build ARPACK from the Rice university source, these are the relevant lines in the makefile:

FC      = /usr/local/mvapich2-2.2-gcc-6.3-cuda-9.2/bin/mpifort
FFLAGS  = -I/usr/local/mvapich2-2.2-gcc-6.3-cuda-9.2/include -fPIC

# Set the parallel fortran compiler
MPILIBS = -L/usr/local/mvapich2-2.2-gcc-6.3-cuda-9.2/lib/libmpifort.so
PFC     = /usr/local/mvapich2-2.2-gcc-6.3-cuda-9.2/bin/mpifort
PFFLAGS = -I/usr/local/mvapich2-2.2-gcc-6.3-cuda-9.2/include -fPIC

There are lines in the makefile to link to BLAS and LAPACK, which we do not use. If we link ARPACK to the installed OpenBLAS shared library, we must also link to the openBLAS shared library in the hll. We wish to avoid this.

When we use ARPACK-NG, we similarly would like for it to avoid any linking to OpenBLAS. However, it linked to the installed shared lib automatically, and we cannot find a CMake option to block this.

Screenshot 2019-05-26 at 2 57 25 PM

At the linking stage of our hll, the executables cannot find the openBLAS functions: Screenshot 2019-05-26 at 2 59 37 PM because we didn't add the openblas library flag that ARPACK-NG automatically linked to.

cpviolator avatar May 26 '19 18:05 cpviolator

Does this fix the problem ? (rebuild from scratch)

>> git diff
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 15f9bb8..884d17b 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -257,8 +257,8 @@ set(arpackutil_STAT_SRCS
 
 
 if (MPI)
-    file(GLOB parpackutil_STAT_SRCS ${arpack_SOURCE_DIR}/PARPACK/UTIL/MPI/*.f)
-    file(GLOB parpacksrc_STAT_SRCS ${arpack_SOURCE_DIR}/dbgini.f ${arpack_SOURCE_DIR}/staini.f ${arpack_SOURCE_DIR}/PARPACK/SRC/MPI/*.f)
+    file(GLOB parpackutil_STAT_SRCS ${arpackutil_STAT_SRCS} ${arpack_SOURCE_DIR}/PARPACK/UTIL/MPI/*.f)
+    file(GLOB parpacksrc_STAT_SRCS ${arpack_SOURCE_DIR}/dbgini.f ${arpack_SOURCE_DIR}/staini.f ${arpack_SOURCE_DIR}/SRC/*stat*.f ${arpack_SOURCE_DIR}/PARPACK/SRC/MPI/*.f)
 endif()
 
 # use -DBUILD_SHARED_LIBS=ON|OFF to control static/shared

fghoussen avatar May 27 '19 20:05 fghoussen

No, it still picks up the installed shared lib:

BLAS_openblas_LIBRARY           */usr/lib/x86_64-linux-gnu/libopenblas.so 

cpviolator avatar May 27 '19 21:05 cpviolator

This problem is unavoidable because ARPACK-NG has omitted the BLAS and LAPACK directories from the original source code. ARPACK is capable of compiling its own BLAS and LAPACK, but ARPACK-NG needs to link to precompiled libraries.

There is some commented out code in your CMakeLists.txt file:

option(SYSTEM_BLAS "Use system BLAS" OFF)
option(SYSTEM_LAPACK "Use system LAPACK" OFF)

Suggesting that at one point there was indeed an option to not use system libraries. However, looking back at the commit history, I see that BLAS and LAPACK were removed 8 years ago... https://github.com/opencollab/arpack-ng/commit/c9d9d92d5108fea1ba617684d55cd8faff1fd146

cpviolator avatar May 27 '19 21:05 cpviolator

Not sure to get the connection with BLAS. Link errors you show are related to arscnd and zstatn and friends : these are arpack (utils) files (and do not involve BLAS). So these ones should disappear: right ? Are there some left ?

fghoussen avatar May 28 '19 07:05 fghoussen

Looking for example to test running debian on Android :D Screenshot_2019-05-31-14-56-16

fghoussen avatar May 31 '19 13:05 fghoussen

May explain/fix the issue. Not sure for pc file. Need https://github.com/opencollab/arpack-ng/issues/214#issuecomment-496301025 anyway. Run from scratch.

Screenshot_2019-06-09-11-58-40

fghoussen avatar Jun 09 '19 10:06 fghoussen

@cpviolator What platform are you on? Are you building arpack-ng as a shared lib or a static lib?

In general, static libs don't record their own dependency info. If your hll is linking to an arpack-ng static lib, you have to specify all of arpack-ng's dependencies. If instead your hll is linking to an arpack-ng shared lib, there's usually no need to do that (unless you're on a platform where shared libs don't have the dependency info either).

turboencabulator avatar Jul 17 '19 05:07 turboencabulator

I just noticed "libparpack.a" in your screenshot. Looks like you've got static libs.

turboencabulator avatar Jul 17 '19 05:07 turboencabulator