sf icon indicating copy to clipboard operation
sf copied to clipboard
verified publisher icon r-spatial

configure: error: libproj or sqlite3 not found in standard or given locations.

Open pvo93 opened this issue 5 years ago • 28 comments

I maintain ~1200 install and maintain R packages on a RHEL7 for which I don't have root access. I've found the best approach is to install all compiled libraries to support R packages in one place. I need to install sf as a dependency for other packages. I've successfully installed gdal, proj7, tiff-4, and geos. I've also installed a newer version of sqlite3 than what was on our system because one of these packages required it (proj7 I believe).

When I got the configure error "libproj or sqlite3 not found in standard or given locations," I initially thought it was because sqlite3 was not in a standard location because the sf config files don't use pkg-config for sqlite3 (although pkg-config is used for proj). However, when I execute the test on the command on the command line, I get the same error whether my PATH is pointing to the default sqlite3 install or mine. So, now I'm not sure if it's a sqlite3 issue at all.

[wa02042@mforgehn5 sf]$ ${CXX} ${CPPFLAGS} -o proj_conf_test proj_conf_test.cpp ${LIBS} -lsqlite3 /tmp/ccuBYW9n.o: In function main': proj_conf_test.cpp:(.text+0x5): undefined reference to proj_context_create' collect2: error: ld returned 1 exit status [wa02042@mforgehn5 sf]$ which sqlite3 /bin/sqlite3 [wa02042@mforgehn5 sf]$ sqlite3 --version 3.7.17 2013-05-20 00:56:22 118a3b35693b134d56ebd780123b7fd6f1497668

[wa02042@mforgehn4 sf]$ g++ -std=gnu++11 -I/usr/local/biotools/r/R-3.6.2/R-dependencies/include -o proj_conf_test proj_conf_test.cpp -lsqlite3 /tmp/ccnTaFWO.o: In function main': proj_conf_test.cpp:(.text+0x5): undefined reference to proj_context_create' collect2: error: ld returned 1 exit status

[wa02042@mforgehn4 sf]$ rpm -E %{rhel} 7 [wa02042@mforgehn4 sf]$ which sqlite3 /usr/local/biotools/ver2/2.0.0/bin/sqlite3 [wa02042@mforgehn4 sf]$ sqlite3 --version 3.33.0 2020-08-14 13:23:32 fca8dc8b578f215a969cd899336378966156154710873e68b3d9ac5881b0ff3f [wa02042@mforgehn4 sf]$ which g++ /usr/local/scl/devtoolset-8/root/usr/bin/g++ [wa02042@mforgehn4 sf]$ g++ --version g++ (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3) Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

I read through the other sqlite3 post where someone questioned if the -sqlite3 flag is necessary at all. I tried hacking the configure.ac file to skip over the test. I was able to get the shared library to compile, but there's a segfault when I try to load it into R.

[wa02042@mforgehn4 R]$ R --vanilla CMD INSTALL -l /usr/local/biotools/rpackages/R-3.6.2-latest sf

  • installing source package ‘sf’ ... files ‘configure’, ‘configure.ac’ have the wrong MD5 checksums ** using staged installation configure: CC: gcc configure: CXX: g++ -std=gnu++11 checking for gdal-config... /usr/local/biotools/ver2/2.0.0/bin/gdal-config checking gdal-config usability... yes configure: GDAL: 3.1.2 checking GDAL version >= 2.0.1... yes checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking gdal.h usability... yes checking gdal.h presence... yes checking for gdal.h... yes checking GDAL: linking with --libs only... no checking GDAL: linking with --libs and --dep-libs... yes checking GDAL: /usr/local/biotools/ver2/2.0.0/share/gdal/pcs.csv readable... no checking GDAL: checking whether PROJ is available for linking:... yes checking GDAL: checking whether PROJ is available fur running:... yes configure: GDAL: 3.1.2 configure: pkg-config proj exists, will use it configure: using proj.h. configure: PROJ: 7.1.0 checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no checking for geos-config... /usr/local/biotools/ver2/2.0.0/bin/geos-config checking geos-config usability... yes configure: GEOS: 3.8.1 checking GEOS version >= 3.4.0... yes checking geos_c.h usability... yes checking geos_c.h presence... yes checking for geos_c.h... yes checking geos: linking with -L/usr/local/biotools/ver2/2.0.0/lib -lgeos_c... yes configure: Package CPP flags: -I/usr/local/biotools/ver2/2.0.0/include -DHAVE_PROJ_H -I/usr/local/biotools/ver2/2.0.0/include -I/usr/local/biotools/ver2/2.0.0/include configure: Package LIBS: -L/usr/local/biotools/ver2/2.0.0/lib -lproj -L/usr/local/biotools/ver2/2.0.0/lib -lgdal -lcrypto -lsqlite3 -lexpat -ljpeg -ltiff -lpng -L/usr/local/biotools/ver2/2.0.0/lib -lproj -lsqlite3 -lz -lpthread -lm -lrt -ldl -lpcre -lcurl -lxml2 -L/usr/local/biotools/ver2/2.0.0/lib -lgeos_c configure: creating ./config.status config.status: creating src/Makevars ** libs make: Nothing to be done for 'all'. installing to /usr/local/biotools/rpackages/R-3.6.2-latest/00LOCK-sf/00new/sf/libs ** R ** demo ** inst ** byte-compile and prepare package for lazy loading in method for ‘dbWriteTable’ with signature ‘"PostgreSQLConnection","character","sf"’: no definition for class “PostgreSQLConnection” in method for ‘dbDataType’ with signature ‘"PostgreSQLConnection","sf"’: no definition for class “PostgreSQLConnection” in method for ‘coerce’ with signature ‘"Spatial","sf"’: no definition for class “Spatial” in method for ‘coerce’ with signature ‘"Spatial","sfc"’: no definition for class “Spatial” in method for ‘coerce’ with signature ‘"sf","Spatial"’: no definition for class “Spatial” in method for ‘coerce’ with signature ‘"sfc","Spatial"’: no definition for class “Spatial” in method for ‘coerce’ with signature ‘"XY","Spatial"’: no definition for class “Spatial” in method for ‘coerce’ with signature ‘"crs","CRS"’: no definition for class “CRS” ** help *** installing help indices *** copying figures ** building package indices ** installing vignettes ** testing if installed package can be loaded from temporary location sh: line 1: 11870 Segmentation fault '/usr/local/biotools/r/R-3.6.2/lib64/R/bin/R' --no-save --slave 2>&1 < '/tmp/RtmpBAXIVh/filed0442428581'

*** caught segfault *** address 0x89156, cause 'memory not mapped'

Traceback: 1: dyn.load(file, DLLpath = DLLpath, ...) 2: library.dynam(lib, package, package.lib) 3: loadNamespace(package, lib.loc) 4: doTryCatch(return(expr), name, parentenv, handler) 5: tryCatchOne(expr, names, parentenv, handlers[[1L]]) 6: tryCatchList(expr, classes, parentenv, handlers) 7: tryCatch({ attr(package, "LibPath") <- which.lib.loc ns <- loadNamespace(package, lib.loc) env <- attachNamespace(ns, pos = pos, deps, exclude, include.only)}, error = function(e) { P <- if (!is.null(cc <- conditionCall(e))) paste(" in", deparse(cc)[1L]) else "" msg <- gettextf("package or namespace load failed for %s%s:\n %s", sQuote(package), P, conditionMessage(e)) if (logical.return) message(paste("Error:", msg), domain = NA) else stop(msg, call. = FALSE, domain = NA)}) 8: library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE) 9: withCallingHandlers(expr, packageStartupMessage = function(c) invokeRestart("muffleMessage")) 10: suppressPackageStartupMessages(library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE)) 11: doTryCatch(return(expr), name, parentenv, handler) 12: tryCatchOne(expr, names, parentenv, handlers[[1L]]) 13: tryCatchList(expr, classes, parentenv, handlers) 14: tryCatch(expr, error = function(e) { call <- conditionCall(e) if (!is.null(call)) { if (identical(call[[1L]], quote(doTryCatch))) call <- sys.call(-4L) dcall <- deparse(call)[1L] prefix <- paste("Error in", dcall, ": ") LONG <- 75L sm <- strsplit(conditionMessage(e), "\n")[[1L]] w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w") if (is.na(w)) w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L], type = "b") if (w > LONG) prefix <- paste0(prefix, "\n ") } else prefix <- "Error : " msg <- paste0(prefix, conditionMessage(e), "\n") .Internal(seterrmessage(msg[1L])) if (!silent && isTRUE(getOption("show.error.messages"))) { cat(msg, file = outFile) .Internal(printDeferredWarnings()) } invisible(structure(msg, class = "try-error", condition = e))}) 15: try(suppressPackageStartupMessages(library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE))) 16: tools:::.test_load_package("sf", "/usr/local/biotools/rpackages/R-3.6.2-latest/00LOCK-sf/00new") An irrecoverable exception occurred. R is aborting now ... ERROR: loading failed

  • removing '€˜/usr/local/biotools/rpackages/R-3.6.2-lates/sf'

Any feedback you could give me here would be greatly appreciated. Thank you.

pvo93 avatar Aug 18 '20 23:08 pvo93

sf doesn't need sqlite3 executable, but the sqlite3 library and header files. For fedora this is packaged in sqlite-devel.

The error you see is frequently seen when certain libraries on your system are installed at more than one place in different versions (e.g. packaged, as well as from source), see here.

edzer avatar Aug 19 '20 05:08 edzer

Thank you for the reply. I tried installing again using the --with-proj-lib flag and with --with-proj-include flags but got the same error. I find the message confusing because even though it says "libproj or sqlite3 not found in standard or given locations." I've found nothing in the documentation or configure files that let's specify the location of sqlite3. Is there an option to specify the location of the sqlite3 libraries?

I've verified that the proj lib is linked to my sqlite3 install. And, I compiled sf.so again with my hacked configure script without letting R test loading it. The resulting sf.so is linked to the proper proj and sqlite3 libs.

[wa02042@mforgehn4 R]$ ldd /usr/local/biotools/rpackages/R-3.6.2-latest/sf/libs/sf.so | grep proj libproj.so.19 => /usr/local/biotools/ver2/2.0.0/lib/libproj.so.19 (0x00002ab53fc6d000) [wa02042@mforgehn4 R]$ ls -l /usr/local/biotools/ver2/2.0.0/lib/libproj.so.19 lrwxrwxrwx 1 wa02042 r-installs 17 Aug 19 12:33 /usr/local/biotools/ver2/2.0.0/lib/libproj.so.19 -> libproj.so.19.1.0 [wa02042@mforgehn4 R]$ ldd /usr/local/biotools/rpackages/R-3.6.2-latest/sf/libs/sf.so | grep sql libsqlite3.so.0 => /usr/local/biotools/ver2/2.0.0/lib/libsqlite3.so.0 (0x00002acaf5a22000)

This is not a standard Fedora box. It's a research node for the Mayo Clinic on which I don't have root access and can't use apt-get so I've compiled everything from source. Any further insight would be greatly appreciated.

pvo93 avatar Aug 19 '20 18:08 pvo93

I think the way forward is to stop breaking ./configure when this test breaks, maybe only giving a message that the test fails. It requires libsqlite3-dev, but that is not required if PROJ and GDAL were installed as binary packages.

edzer avatar Aug 19 '20 22:08 edzer

I am having a similar problem

checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /data/software/gdal/3.1.2_hdf4/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... yes
configure: GDAL: 3.1.2
configure: pkg-config proj exists, will use it
configure: using proj.h.
configure: PROJ: 7.1.0
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
configure: error: libproj or sqlite3 not found in standard or given locations.
ERROR: configuration failed for package ‘sf’
* removing ‘/data/home/sserbin/R/library/sf’

The downloaded source packages are in
	‘/tmp/Rtmpq9oxYT/downloaded_packages’
Warning message:
In install.packages("sf", configure.args = c("--with-proj-include=/data/software/proj/7.1.0/include/",  :
  installation of package ‘sf’ had non-zero exit status
> q()

Is it possible to use a flag like --with-sqlite3-lib? like with proj? For reference here is what I tried

install.packages('sf', configure.args=c('--with-proj-include=/data/software/proj/7.1.0/include/','--with-proj-lib=/data/software/proj/7.1.0/lib/', '--with-proj=/data/software/proj/7.1.0') )

serbinsh avatar Oct 15 '20 20:10 serbinsh

Feel free to submit a PR - I see this is a problem, someone has to solve it.

edzer avatar Oct 16 '20 07:10 edzer

Hi, I have had this problem too. I need to specify sqlite3 library paths because I can't install it on standard paths. Specific parameters would be useful.

Anyway i managed this issue using environment vauables: LIBS, LDFLAGS, CPPFLAGS anc CFLAGS to specify non standard paths for the compiler.

Kind Regards

grimli avatar Oct 17 '20 10:10 grimli

Hi What is the solution to this ? What env variables are you using for sql lite ? I am have setup the following variable (to the correct file locations not list) for both sqlite3 and Proj7

LD_LIBRARY_PATH LIBRARY_PATH PATH MANPATH C_INCLUDE_PATH CPLUS_INCLUDE_PAT INCLUDE PKG_CONFIG_PATH " CMAKE_PREFIX_PATH

This is my error

checking gdal.h usability... yes checking gdal.h presence... yes checking for gdal.h... yes checking GDAL: linking with --libs only... no checking GDAL: linking with --libs and --dep-libs... yes checking GDAL: /mnt/opt/Centos7/hpc/spack/spack_Oct_2020/opt/spack/linux-centos7-broadwell/gcc-7.3.1/gdal-3.2.0-7juusgnasjeppoqamonw253tvgpeaafs/share/gdal/pcs.csv readable... no checking GDAL: checking whether PROJ is available for linking:... yes checking GDAL: checking whether PROJ is available fur running:... yes configure: GDAL: 3.2.0 configure: pkg-config proj exists, will use it configure: using proj.h. configure: PROJ: 7.1.0 checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no configure: error: libproj or sqlite3 not found in standard or given locations. ERROR: configuration failed for package ���sf���

removing ���/mnt/opt/Centos7/hpc/spack/spack_Oct_2020/opt/spack/linux-centos7-broadwell/gcc-7.3.1/r-4.0.3-tu34p762h7igt6pdf7wlfm4fnaxmxyne/rlib/R/library/sf��� The downloaded source packages are in ���/mnt/gpfs/tmp/R/RtmpDek6az/downloaded_packages��� Updating HTML index of packages in '.Library' Making 'packages.html' ... done Warning message: In install.packages("sf") : installation of package ���sf��� had non-zero exit status

rrobinson-RMS avatar Mar 16 '21 14:03 rrobinson-RMS

@rrobinson-RMS

I'm loading environment modules (lmod) within our ldpaths file within the R install to load sqlite and proj modules, both prepending PATH, CPATH, LD_LIBRARY_PATH, and PKG_CONFIG_PATH

jekriske-lilly avatar Mar 16 '21 14:03 jekriske-lilly

Hi Jeff, Thanks for your answer I used to work at Lilly too some years ago !

I am using standard Linux environment modules ie module load ... Then I after loading all modules just running R and installing from there.

the only variable you have listed above I am missing is CPATH. Which I believe is the same as C_INCLUDE_PATH. I've added it to my Proj + Sqlite modules.

but no still luck ! cheers

rrobinson-RMS avatar Mar 16 '21 14:03 rrobinson-RMS

Are you also loading the gdal module and can you confirm that both gdal and proj were compiled with the sqlite version you're using?

For proj, I'm using 7.2.0, but I forget the minimum version needed.

jekriske-lilly avatar Mar 16 '21 14:03 jekriske-lilly

Fixed it, my GEOS module was not the one referenced in my GDAL module.

The Error message was completely unrelated unfortunately.

Thanks for your help jeff as mentioning GDAL made me start looking along the rest of the chain.

rrobinson-RMS avatar Mar 17 '21 12:03 rrobinson-RMS

I have a similar problem. I'm thinking it is because I'm on the cluster and load the module for dgal, proj and geos sperately. gdal/gcc/2.4.1 proj/gcc/6.0.0 geos/gcc/3.7.2

there is no version problems, but I got the same error with @serbinsh

  • installing source package 'sf' ... ** using staged installation configure: CC: gcc configure: CXX: g++ -std=gnu++11 checking for gdal-config... /cm/shared/apps/gdal/gcc/2.4.1/bin/gdal-config checking gdal-config usability... yes configure: GDAL: 2.4.1 checking GDAL version >= 2.0.1... yes checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /usr/bin/grep checking for egrep... /usr/bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking gdal.h usability... yes checking gdal.h presence... yes checking for gdal.h... yes checking GDAL: linking with --libs only... yes checking GDAL: /cm/shared/apps/gdal/gcc/2.4.1/share/gdal/pcs.csv readable... yes checking GDAL: checking whether PROJ is available for linking:... yes checking GDAL: checking whether PROJ is available fur running:... yes configure: GDAL: 2.4.1 configure: pkg-config proj exists, will use it configure: using proj.h. configure: PROJ: 6.0.0 checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no configure: error: libproj or sqlite3 not found in standard or given locations. ERROR: configuration failed for package 'sf'
  • removing '/home2/s205236/R/x86_64-pc-linux-gnu-library/4.0/sf' Warning message: In i.p(...) : installation of package '/tmp/RtmpDFwXL3/file1defe226a92cd/sf_1.0-0.tar.gz' had non-zero exit status

how shall I identify the directory for those modules and sqlite?

ice4prince avatar Jun 07 '21 21:06 ice4prince

I am having a similar problem

checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /data/software/gdal/3.1.2_hdf4/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... yes
configure: GDAL: 3.1.2
configure: pkg-config proj exists, will use it
configure: using proj.h.
configure: PROJ: 7.1.0
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
configure: error: libproj or sqlite3 not found in standard or given locations.
ERROR: configuration failed for package ‘sf’
* removing ‘/data/home/sserbin/R/library/sf’

The downloaded source packages are in
	‘/tmp/Rtmpq9oxYT/downloaded_packages’
Warning message:
In install.packages("sf", configure.args = c("--with-proj-include=/data/software/proj/7.1.0/include/",  :
  installation of package ‘sf’ had non-zero exit status
> q()

Is it possible to use a flag like --with-sqlite3-lib? like with proj? For reference here is what I tried

install.packages('sf', configure.args=c('--with-proj-include=/data/software/proj/7.1.0/include/','--with-proj-lib=/data/software/proj/7.1.0/lib/', '--with-proj=/data/software/proj/7.1.0') )

I have the same issue. Have you succeeded? How did you find these directories?

ice4prince avatar Jun 07 '21 21:06 ice4prince

Fixed it, my GEOS module was not the one referenced in my GDAL module.

The Error message was completely unrelated unfortunately.

Thanks for your help jeff as mentioning GDAL made me start looking along the rest of the chain.

Fixed it, my GEOS module was not the one referenced in my GDAL module.

The Error message was completely unrelated unfortunately.

Thanks for your help jeff as mentioning GDAL made me start looking along the rest of the chain.

@rrobinson-RMS Hi Robinson, how did you solve your problem. I have the same issue with you. I'm on cluster too. I have load gdal/gcc/2.4.1 proj/gcc/6.0.0 geos/gcc/3.7.2 and sqlite can be accessed, how did you fix your issue?

ice4prince avatar Jun 14 '21 22:06 ice4prince

I'm facing the same issue so I'm re-posting on this thread as it doesn't seem to have been resolved yet.

I'm running R 4.0.3 on a cluster and load gdal (3.2.0), geos (3.8.1), proj (7.1.0) and sqlite (3.33.0) modules before launching R and trying to install sf package. I get the same error as @ice4prince @rrobinson-RMS and others reported:

  • installing source package ‘sf’ ... ** package ‘sf’ successfully unpacked and MD5 sums checked ** using staged installation configure: CC: /home/support/spack/spack-0.16.1-compilers_and_lmod/spack/opt/spack/linux-scientific7-x86_64/gcc-4.8.5/gcc-9.3.0-wkarmehqv332dvhtly67cpfchgyzgv65/bin/gcc configure: CXX: /home/support/spack/spack-0.16.1-compilers_and_lmod/spack/opt/spack/linux-scientific7-x86_64/gcc-4.8.5/gcc-9.3.0-wkarmehqv332dvhtly67cpfchgyzgv65/bin/g++ checking for gdal-config... /home/support/spack/spack-0.16.1/spack/opt/spack/linux-scientific7-x86_64/gcc-9.3.0/gdal-3.2.0-6iih24h46ircshgzjsk6rwrztfrlp5xk/bin/gdal-config checking gdal-config usability... yes configure: GDAL: 3.2.0 checking GDAL version >= 2.0.1... yes checking for gcc... /home/support/spack/spack-0.16.1-compilers_and_lmod/spack/opt/spack/linux-scientific7-x86_64/gcc-4.8.5/gcc-9.3.0-wkarmehqv332dvhtly67cpfchgyzgv65/bin/gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether /home/support/spack/spack-0.16.1-compilers_and_lmod/spack/opt/spack/linux-scientific7-x86_64/gcc-4.8.5/gcc-9.3.0-wkarmehqv332dvhtly67cpfchgyzgv65/bin/gcc accepts -g... yes checking for /home/support/spack/spack-0.16.1-compilers_and_lmod/spack/opt/spack/linux-scientific7-x86_64/gcc-4.8.5/gcc-9.3.0-wkarmehqv332dvhtly67cpfchgyzgv65/bin/gcc option to accept ISO C89... none needed checking how to run the C preprocessor... /home/support/spack/spack-0.16.1-compilers_and_lmod/spack/opt/spack/linux-scientific7-x86_64/gcc-4.8.5/gcc-9.3.0-wkarmehqv332dvhtly67cpfchgyzgv65/bin/gcc -E checking for grep that handles long lines and -e... /usr/bin/grep checking for egrep... /usr/bin/grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking gdal.h usability... yes checking gdal.h presence... yes checking for gdal.h... yes checking GDAL: linking with --libs only... yes checking GDAL: /home/support/spack/spack-0.16.1/spack/opt/spack/linux-scientific7-x86_64/gcc-9.3.0/gdal-3.2.0-6iih24h46ircshgzjsk6rwrztfrlp5xk/share/gdal/pcs.csv readable... no checking GDAL: checking whether PROJ is available for linking:... yes checking GDAL: checking whether PROJ is available fur running:... yes configure: GDAL: 3.2.0 configure: pkg-config proj exists, will use it configure: using proj.h. configure: PROJ: 7.1.0 checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no configure: error: libproj or sqlite3 not found in standard or given locations. ERROR: configuration failed for package ‘sf’
  • removing ‘/home/users/blanchap/myRlib/sf’

The downloaded source packages are in ‘/tmp/RtmpQvlvzR/downloaded_packages’ Warning message: In install.packages("sf") : installation of package ‘sf’ had non-zero exit status

Does anyone know how to solve this issue?

blanchap avatar Sep 29 '21 13:09 blanchap

I have the same issue:

* installing *source* package ‘sf’ ...
** package ‘sf’ successfully unpacked and MD5 sums checked
** using staged installation
configure: CC: /apps/jasmin/jaspy/miniconda_envs/jaspy3.7/m3-4.9.2/envs/jaspy3.7-m3-4.9.2-r20210320/bin/x86_64-conda-linux-gnu-cc
configure: CXX: /apps/jasmin/jaspy/miniconda_envs/jaspy3.7/m3-4.9.2/envs/jaspy3.7-m3-4.9.2-r20210320/bin/x86_64-conda-linux-gnu-c++
checking for gdal-config... /apps/jasmin/jaspy/miniconda_envs/jaspy3.7/m3-4.9.2/envs/jaspy3.7-m3-4.9.2-r20210320/bin/gdal-config
checking gdal-config usability... yes
configure: GDAL: 3.1.4
checking GDAL version >= 2.0.1... yes
checking for x86_64-conda-linux-gnu-gcc... /apps/jasmin/jaspy/miniconda_envs/jaspy3.7/m3-4.9.2/envs/jaspy3.7-m3-4.9.2-r20210320/bin/x86_64-conda-linux-gnu-cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /apps/jasmin/jaspy/miniconda_envs/jaspy3.7/m3-4.9.2/envs/jaspy3.7-m3-4.9.2-r20210320/bin/x86_64-conda-linux-gnu-cc accepts -g... yes
checking for /apps/jasmin/jaspy/miniconda_envs/jaspy3.7/m3-4.9.2/envs/jaspy3.7-m3-4.9.2-r20210320/bin/x86_64-conda-linux-gnu-cc option to accept ISO C89... none needed
checking how to run the C preprocessor... /apps/jasmin/jaspy/miniconda_envs/jaspy3.7/m3-4.9.2/envs/jaspy3.7-m3-4.9.2-r20210320/bin/x86_64-conda-linux-gnu-cpp
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking gdal.h usability... yes
checking gdal.h presence... yes
checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /apps/jasmin/jaspy/miniconda_envs/jaspy3.7/m3-4.9.2/envs/jaspy3.7-m3-4.9.2-r20210320/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... yes
configure: GDAL: 3.1.4
configure: pkg-config proj exists, will use it
configure: using proj.h.
configure: PROJ: 7.1.1
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
configure: error: libproj or sqlite3 not found in standard or given locations.
ERROR: configuration failed for package ‘sf’
* removing ‘/home/users/simrol/R/x86_64-conda_cos6-linux-gnu-library/4.0/sf’

The downloaded source packages are in
        ‘/tmp/Rtmpe9kl1n/downloaded_packages’
Warning message:
In install.packages("sf") :
  installation of package ‘sf’ had non-zero exit status

Session info

> sessionInfo()
R version 4.0.2 (2020-06-22)
Platform: x86_64-conda_cos6-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: /apps/jasmin/jaspy/miniconda_envs/jaspy3.7/m3-4.9.2/envs/jaspy3.7-m3-4.9.2-r20210320/lib/libmkl_rt.so

locale:
 [1] LC_CTYPE=en_GB.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_GB.UTF-8        LC_COLLATE=en_GB.UTF-8
 [5] LC_MONETARY=en_GB.UTF-8    LC_MESSAGES=en_GB.UTF-8
 [7] LC_PAPER=en_GB.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

loaded via a namespace (and not attached):
[1] compiler_4.0.2 tools_4.0.2    curl_4.3.2     remotes_2.4.1

Back in the terminal I've got proj and sqlite3

[simrol@sci2 ~]$ proj
Rel. 7.1.1, September 1st, 2020
usage: proj [-bdeEfiIlmorsStTvVwW [args]] [+opt[=arg] ...] [file ...]

[simrol@sci2 ~]$ sqlite3
SQLite version 3.34.0 2020-12-01 16:14:00
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite>

simonrolph avatar Oct 08 '21 10:10 simonrolph

Back in the terminal I've got proj and sqlite3

the development libraries of sqlite3 are needed, not the executable. In general, this is some kind of conda install, and I have no clue what is going on there.

edzer avatar Oct 08 '21 11:10 edzer

After much tedious trial and error, I think I may have a solution for those still running into this error. Two steps are necessary:

  1. Compiling the dependency libraries with linking configured
  2. Adding environment paths

I am using proj v8.1.1, sqlite3 v3.33.0 and gdal-config v3.3.2.

When compiling proj, add /lib/pkgconfig/sqlite3.pc to PKG_CONFIG_PATH otherwise ./configure will fail When compiling gdal, add --with-python and --with-proj flags to ./configure. Finally, add to LD_LIBRARY_PATH (also LIBRARY_PATH, CPATH, PKG_CONFIG_PATH but not as essential) the lib directory for proj, sqlite3 and gdal.

Then I got the following output:

* installing *source* package ‘sf’ ...
** package ‘sf’ successfully unpacked and MD5 sums checked
** using staged installation
configure: CC: /gsc/software/linux-x86_64-centos7/gcc-7.2.0/bin/gcc
configure: CXX: /gsc/software/linux-x86_64-centos7/gcc-7.2.0/bin/g++ -std=gnu++11
checking for gdal-config... /home/dchiu/lib/gdal-3.3.2/bin/gdal-config
checking gdal-config usability... yes
configure: GDAL: 3.3.2
checking GDAL version >= 2.0.1... yes
checking for gcc... /gsc/software/linux-x86_64-centos7/gcc-7.2.0/bin/gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /gsc/software/linux-x86_64-centos7/gcc-7.2.0/bin/gcc accepts -g... yes
checking for /gsc/software/linux-x86_64-centos7/gcc-7.2.0/bin/gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... /gsc/software/linux-x86_64-centos7/gcc-7.2.0/bin/gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking gdal.h usability... yes
checking gdal.h presence... yes
checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /home/dchiu/lib/gdal-3.3.2/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... yes
configure: GDAL: 3.3.2
configure: pkg-config proj exists, will use it
configure: using proj.h.
configure: PROJ: 8.1.1
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... yes
checking for geos-config... /usr/bin/geos-config
checking geos-config usability... yes
configure: GEOS: 3.4.2
checking GEOS version >= 3.4.0... yes
checking geos_c.h usability... yes
checking geos_c.h presence... yes
checking for geos_c.h... yes
checking geos: linking with -L/usr/lib64 -lgeos_c... yes
configure: Package CPP flags:  -I/home/dchiu/lib/proj/include   -DHAVE_PROJ_H -I/home/dchiu/lib/gdal-3.3.2/include -I/usr/include
configure: Package LIBS: -L/home/dchiu/lib/proj/lib -lproj     -L/home/dchiu/lib/gdal-3.3.2/lib -lgdal -L/usr/lib64 -lgeos_c
configure: creating ./config.status
config.status: creating src/Makevars

P.S. Using gdal v2.3.2 has the /share/gdal/pcs.csv data file although not needed to install sf

dchiu911 avatar Nov 30 '21 23:11 dchiu911

Another note for anyone still experiencing issues. Check ldd for the gdal libraries after compilation. While testing we had no trouble, but in production, gdal would add system library paths to the rpath overriding the custom path to sqlite3 libraries (which it also correctly had, but later in the rpath string). This caused everything down the chain to fail.

jekriske-lilly avatar Dec 01 '21 12:12 jekriske-lilly

Very very strangely, coming back to this, I was able to get the compilation to work with

PKG_CONFIG_PATH=/data/software/proj/7.2.1/lib/pkgconfig/ R
install.packages('sf', configure.args=c('--with-proj-include=/data/software/proj/7.2.1/include/','--with-proj-lib=/data/software/proj/7.2.1/lib/', '--with-proj=/data/software/proj/7.2.1') )

yet

configure: GDAL: 3.4.2
configure: pkg-config proj exists, will use it
configure: using proj.h.
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... yes
checking for geos-config... /data/software/geos/3.10.2-gcc850/bin/geos-config
checking geos-config usability... yes
configure: GEOS: 3.10.2
checking GEOS version >= 3.4.0... yes
checking geos_c.h usability... yes
checking geos_c.h presence... yes
checking for geos_c.h... yes
checking geos: linking with -L/data/software/geos/3.10.2-gcc850/lib64 -lgeos_c... yes
configure: Package CPP flags:  -I/data/software/proj/7.2.1/include/ -DHAVE_PROJ_H -I/data/software/gdal/3.4.2/include -I/data/software/geos/3.10.2-gcc850/include
configure: Package LIBS: -L/data/software/proj/7.2.1/lib/  -lproj  -L/data/software/gdal/3.4.2/lib -lgdal -L/data/software/geos/3.10.2-gcc850/lib64 -lgeos_c
configure: creating ./config.status
config.status: creating src/Makevars
configure: WARNING: unrecognized options: --with-proj
** libs

even with the warming "configure: WARNING: unrecognized options: --with-proj ** libs"

but when i try just

PKG_CONFIG_PATH=/data/software/proj/7.2.1/lib/pkgconfig/ R
install.packages('sf', configure.args=c('--with-proj-include=/data/software/proj/7.2.1/include/', '--with-proj-lib=/data/software/7.2.1/lib/'))

I get the original error I posted earlier above???

serbinsh avatar Mar 23 '22 13:03 serbinsh 

> install.packages("rnaturalearth")
Installing package into ‘/opt/homebrew/lib/R/4.2/site-library’
(as ‘lib’ is unspecified)
also installing the dependency ‘sf’

trying URL 'https://cloud.r-project.org/src/contrib/sf_1.0-7.tar.gz'
Content type 'application/x-gzip' length 3529597 bytes (3.4 MB)
==================================================
downloaded 3.4 MB

trying URL 'https://cloud.r-project.org/src/contrib/rnaturalearth_0.1.0.tar.gz'
Content type 'application/x-gzip' length 193162 bytes (188 KB)
==================================================
downloaded 188 KB

* installing *source* package ‘sf’ ...
** package ‘sf’ successfully unpacked and MD5 sums checked
** using staged installation
configure: CC: clang
configure: CXX: clang++ -std=gnu++11
checking for gdal-config... /opt/homebrew/bin/gdal-config
checking gdal-config usability... yes
configure: GDAL: 3.5.0
checking GDAL version >= 2.0.1... yes
checking for gcc... clang
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether clang accepts -g... yes
checking for clang option to accept ISO C89... none needed
checking how to run the C preprocessor... clang -E
checking for grep that handles long lines and -e... /opt/homebrew/bin/ggrep
checking for egrep... /opt/homebrew/bin/ggrep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking gdal.h usability... yes
checking gdal.h presence... yes
checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /opt/homebrew/Cellar/gdal/3.5.0_1/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... yes
configure: GDAL: 3.5.0
configure: pkg-config proj exists, will use it
configure: using proj.h.
configure: PROJ: 9.0.0
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
configure: error: libproj or sqlite3 not found in standard or given locations.
ERROR: configuration failed for package ‘sf’
* removing ‘/opt/homebrew/lib/R/4.2/site-library/sf’
ERROR: dependency ‘sf’ is not available for package ‘rnaturalearth’
* removing ‘/opt/homebrew/lib/R/4.2/site-library/rnaturalearth’

The downloaded source packages are in
	‘/private/var/folders/1j/rd7tw1xx1cn4t8xcfh4rf0bh0000gn/T/Rtmp2KjpU1/downloaded_packages’
Warning messages:
1: In install.packages("rnaturalearth") :
  installation of package ‘sf’ had non-zero exit status
2: In install.packages("rnaturalearth") :
  installation of package ‘rnaturalearth’ had non-zero exit status

Same problem on macOS 12.

> sessionInfo()
R version 4.2.0 (2022-04-22)
Platform: aarch64-apple-darwin21.3.0 (64-bit)
Running under: macOS Monterey 12.4

Matrix products: default
BLAS:   /opt/homebrew/Cellar/openblas/0.3.20/lib/libopenblasp-r0.3.20.dylib
LAPACK: /opt/homebrew/Cellar/r/4.2.0/lib/R/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] compiler_4.2.0 tools_4.2.0

pkg-config gives the path to the brewed gdal and proj and the system sqlite3.

pkg-config sqlite3 gdal proj --libs
-L/opt/homebrew/Cellar/gdal/3.5.0_1/lib -L/opt/homebrew/Cellar/proj/9.0.0_1/lib -lsqlite3 -lgdal -lproj

mys721tx avatar Jun 13 '22 07:06 mys721tx

@mys721tx please see section: "Installing packages linking to PROJ, GDAL or GEOS" on https://cran.r-project.org/view=Spatial. Unless you meet those criteria for a source install, use binary installs. If you do meet them, you will yourself understand that upstream external dependencies must be met first.

rsbivand avatar Jun 13 '22 07:06 rsbivand

@rsbivand Please see the printout for pkg-config in the bottom. The external dependencies are installed.

mys721tx avatar Jun 13 '22 08:06 mys721tx

@mys721tx do you need to install from source? If not, install sf binary from CRAN. Otherwise you have to figure out yourself how to pass correct configure arguments to install.packages("sf").

rsbivand avatar Jun 13 '22 08:06 rsbivand

Yes, for my particular setup the binary package does not work since the homebrew .

I did some digging and it turned out that Homebrew's libtiff depends on jpeg. This causes problem with configure.ac since jpeg is not in LIBS. This is the reason --with-proj-lib=/opt/homebrew/lib/ makes it work on macOS.

ld: library not found for -ljpeg
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I will work on a PR and hopefully it will fix the problem on macOS.

mys721tx avatar Jun 13 '22 09:06 mys721tx

Why can you not simply install sf binay and then other packages as you wish?

Does the specific version of GDAL, PROJ or GEOS matter for your use case?

There is no point creating any homebrew PR, homebrew is not a CRAN-supported installation approach.

rsbivand avatar Jun 13 '22 09:06 rsbivand

There is a more fundamental problem at hand. The error message whether PROJ and sqlite3 are available for linking is misleading since sqlite3 is shipped with all macOS. In fact jpeg is the one missing as libtiff does not correctly report its dependency in pkg-config. After I corrected libtiff-4.pc, sf successfully builds without any user intervention. I have submitted a PR to libtiff-4 and hopefully it will propagate down the line.

mys721tx avatar Jun 13 '22 10:06 mys721tx

Homebrew is advised against across R-macOS because of the present danger of mixing build train components (slightly different versions of compilers and linkers, leading to unreproducible errors. CRAN binary packages are built static to ensure that everyone is seeing the same software. All should use CRAN binaries only, unless they are package developers, and these should use the static binary upstream libraries built using the same clang/clang++ versions used for the software into which packages are being dynamically loaded. It may be that your PR to libtiff-4.pc mirrors work already done on the R-macOS side, on the Windows side I know that published pkgconfig files are often wrong, especially for static builds, so they are often hot-patched.

rsbivand avatar Jun 13 '22 18:06 rsbivand

If you are on apple silicon (M series) processor and also used homebrew to install the library, try use this command, it worked for me.

updated on 2023-04-24

For package sf

install.packages('sf', configure.args = configure.args = '--with-gdal-config=/opt/homebrew/bin/gdal-config --with-geos-config=/opt/homebrew/Cellar/geos/3.11.2/bin/geos-config --with-proj-include=/opt/homebrew/Cellar/proj/9.2.0/include --with-proj-lib=/opt/homebrew/Cellar/proj/9.2.0/lib', configure.vars = 'GDAL_DATA=/opt/homebrew/opt/gdal/share/gdal/')

Also for package terra

install.packages('terra', configure.args = '--with-gdal-config=/opt/homebrew/bin/gdal-config --with-geos-config=/opt/homebrew/Cellar/geos/3.11.2/bin/geos-config --with-proj-include=/opt/homebrew/Cellar/proj/9.2.0/include --with-proj-lib=/opt/homebrew/Cellar/proj/9.2.0/lib', configure.vars = 'GDAL_DATA=/opt/homebrew/opt/gdal/share/gdal/')

sepmein avatar Oct 12 '22 14:10 sepmein

I'm encountering the same issue. I can use the binary, of course. But I would like to modify the source code to make a PR. This cannot be done without being able to run the package locally. R CMD build and devtools::load_all() fails with

checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
configure: error: libproj or sqlite3 not found in standard or given locations.

proj and sqlite have been installed via homebrew and are available.

Is there other documentation that I can refer to?

JosiahParry avatar Feb 22 '23 20:02 JosiahParry