r-spatial
Installing sf from compiled GDAL, PROJ and GEOS in Red Hat Enterprise Linux RHEL 8.9 (no admin user)
LSB Version: :core-4.1-amd64:core-4.1-noarch
Distributor ID: RedHatEnterprise
Description: Red Hat Enterprise Linux release 8.9 (Ootpa)
Release: 8.9
Codename: Ootpa
----
R version 4.3.3 (2024-02-29) -- "Angel Food Cake"
I need to install sf from GitHub to allow proper reading of HDF5 files. I don't have an admin user. IT support can help me to install from repositories but not to compile from source.
IT support updated all the dependencies. Library names: udunits2-devel, libcurl-devel, libxml2-devel, libgdal-dev, proj-dev, sqlite-devel, geos-config. The versions of the libraries were at that point as shown bellow:
library(sf)
#> Linking to GEOS 3.7.2, GDAL 3.0.4, PROJ 6.3.2; sf_use_s2() is TRUE
library(stars)
#> Loading required package: abind
library(terra)
#> terra 1.7.74
Nonetheless, the sf package was unable to read the coordinates from HDF5 files.
So, I started the process of complying from sources. I managed to compile everything from 1 to 11. I got error only on 12 (installing sf from GitHub)
-
Install the dependencies (
udunits2-devel,libcurl-devel,libxml2-devel) (Admin IT user) -
Compile SQLite3 Download: https://www.sqlite.org/2024/sqlite-autoconf-3450300.tar.gz
mkdir sqlite
tar xvfz sqlite-autoconf-3450300.tar.gz
cd sqlite-autoconf-3450300
./configure
make
make install DESTDIR=~/sqlite
- Edit
.bashrcfile to add or edit environment variables
export PATH="/home/myuser/sqlite:$PATH"
export SQLite3_INCLUDE_DIR=/home/myuser/sqlite/usr/local/include
export SQLite3_LIBRARY=/home/myuser/sqlite/usr/local/lib/libsqlite3.so
export LD_LIBRARY_PATH="/home/myuser/sqlite/usr/local/lib:$LD_LIBRARY_PATH"
- Compile GTest
Procedure: here
mkdir ~/work-tmp
cd work-tmp/
git clone https://github.com/google/googletest.git
cd googletest/
ls -la
git tag -l
git checkout v1.14.0
mkdir build
cd build
cmake ..
make
mkdir ~/googletest
make install DESTDIR=~/googletest
- Edit
.bashrc
export PATH="~/googletest/usr/local/include:$PATH"
export PATH="~/googletest/usr/local/lib64:$PATH"
- Compiling PROJ
Download: here Procedure: here
mkdir proj940
tar xvf proj-9.4.0.tar.gz
cd proj-9.4.0
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=$HOME/proj940 .. -DCMAKE_CXX_STANDARD=14 -DCMAKE_CXX_STANDARD_REQUIRED=ON
cmake --build .
cmake --build . --target install
- Edit
.bashrc
export PROJ_INCLUDE_DIR=/home/myuser/proj940/include
export PROJ_LIBRARY_RELEASE=/home/myuser/proj940/lib64/libproj.so
- Compiling GEOS
The installer: here The procedure: here
mkdir geos3121
tar xvfj geos-3.12.1.tar.bz2
cd geos-3.12.1
mkdir _build
cd _build
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$HOME/geos3121 .. -DCMAKE_CXX_STANDARD=14 -DCMAKE_CXX_STANDARD_REQUIRED=ON
make
ctest
make install
9)Edit .bashrc
export GEOS_INCLUDE_DIR=/home/myuser/geos3121/include
export GEOS_LIBRARY=/home/myuser/geos3121/lib64/libgeos_c.so
export GEOS_DIR=/home/myuser/geos3121
- Compiling GDAL
Download: here Procedure: here
mkdir gdal385
tar xvf gdal-3.8.5.tar.gz
cd gdal-3.8.5
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$HOME/gdal385 .. -DCMAKE_CXX_STANDARD=14 -DCMAKE_CXX_STANDARD_REQUIRED=ON -DPROJ_INCLUDE_DIR=$PROJ_INCLUDE_DIR -DPROJ_LIBRARY_RELEASE=$PROJ_LIBRARY_RELEASE -DGEOS_INCLUDE_DIR=$GEOS_INCLUDE_DIR -DGEOS_LIBRARY=$GEOS_LIBRARY -DSQLite3_INCLUDE_DIR=$SQLite3_INCLUDE_DIR -DSQLite3_LIBRARY=$SQLite3_LIBRARY -DGDAL_USE_SQLITE3=ON
cmake --build .
cmake --build . --target install
The outcome of the first cmake is
cmake_gdal_outcome.txt. There is a warning:
CMake Warning:
Manually-specified variables were not used by the project:
GEOS_INCLUDE_DIR
GEOS_LIBRARY
-
edit
.bashrcexport GDAL_LIBRARY_PATH=/home/myuser/gdal385/lib64/libgdal.so -
Install
sffrom GitHub using non-standard locations (compiled folders)
remotes::install_github("r-spatial/sf", configure.args = "--with-gdal-config=/home/myuser/gdal385/bin/gdal-config --with-geos-config=/home/myuser/geos3121/bin/geos-config --with-proj-include=/home/myuser/proj940/include/ --with-proj-lib=/home/myuser/proj940/lib64/")
The outcome of sf GitHub install is sf_install_github.txt. There are some issues related to:
...
libgdal.so: undefined reference to
...
libgeos_c.so: undefined reference to
...
collect2: error: ld returned 1 exit status
configure: Install failure: compilation and/or linkage problems.
configure: error: GDALAllRegister not found in libgdal.
ERROR: configuration failed for package ‘sf’
...
Any advice will be appreciated.
Best
I complemented the LD_LIBRARY_PATH inside .bashrc:
export LD_LIBRARY_PATH="/home/myuser/sqlite/usr/local/lib:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH="~/googletest/usr/local/lib64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH="/home/myuser/proj940/lib64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH="/home/myuser/geos3121/lib64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH="/home/myuser/gdal385/lib64:$LD_LIBRARY_PATH"
After running
remotes::install_github("r-spatial/sf", configure.args = "--with-gdal-config=/home/myuser/gdal385/bin/gdal-config --with-geos-config=/home/myuser/geos3121/bin/geos-config --with-proj-include=/home/myuser/proj940/include/ --with-proj-lib=/home/myuser/proj940/lib64/")
the errors are shown below
configure: error: libproj or sqlite3 not found in standard or given locations.
Then I found this:
https://github.com/r-spatial/sf#multiple-gdal-geos-andor-proj-versions-on-your-system
Let's see if I can solve it!
This person tried setting the variable inside R with some success: https://github.com/r-spatial/sf/issues/844#issuecomment-654477840
I had to add some additional details, because after implementing his solution, I got the next error:
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘sf’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/home/myuser/R/x86_64-pc-linux-gnu-library/4.4/00LOCK-sf/00new/sf/libs/sf.so':
/home/myuser/gdal385/lib64/libgdal.so.34: undefined symbol: sqlite3_total_changes64
These are the steps to solve my issue:
- IT (admin user) support removed
proj(binaries or headers/devel) from the system. - According to IT support (admin user): The
SQLitelibrary (not the devel package) was not removed because it was required forRstudio. - Libraries were loaded inside R by adding next lines to
.Rprofile
Important: The order of loading libraries matters!
Note the use of the variable CPATH and the dynamic load of sqlite
Sys.setenv("LD_LIBRARY_PATH" = "/home/muyser/gdal385/lib64:/home/muyser/geos3121/lib64:/home/myuser/proj940/lib64:~/googletest/usr/local/lib64:/home/myuser/sqlite/usr/local/lib")
dyn.load("/home/myuser/geos3121/lib64/libgeos.so.3.12.1")
dyn.load("/home/myuser/geos3121/lib64/libgeos_c.so")
Sys.setenv("CPATH" = "/home/myuser/proj940/include:/home/myuser/sqlite/usr/local/include")
dyn.load("/home/myuser/sqlite/usr/local/lib/libsqlite3.so")
dyn.load("/home/myuser/sqlite/usr/local/lib/libsqlite3.so.0.8.6")
dyn.load("/home/myuser/proj940/lib64/libproj.so.25.9.4.0")
dyn.load("/home/myuser/proj940/lib64/libproj.so")
Then I was able to install from GitHub using:
remotes::install_github("r-spatial/sf", configure.args = "--with-gdal-config=/home/myuser/gdal385/bin/gdal-config --with-geos-config=/home/myuser/geos3121/bin/geos-config --with-proj-include=/home/myuser/proj940/include/ --with-proj-lib=/home/myuser/proj940/lib64/")
Extra Note: If you get the next error when compiling gdal, it was related to extra characters (:) in the environment variables. It was solved by removing the extra ":" at the end of some environment variables defined inside .bashrc.
[93%] Built target gcore_mdreader
CMakeFiles/GDAL.dir/build.make:2457: *** target pattern contains no '%'. Stop.
make[1]: *** [CMakeFiles/Makefile2:4697: CMakeFiles/GDAL.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
Congrats on getting this done, and I'd agree that this is a very hard and painful procedure.
Thank you!. Please let me know if you have any additional advice or different procedures for making this easier!