Timout downloading ICU at stringi installation

[MatMatt]

I face a timeout issue during the installation of stringi. The issue seems to be connected to a timeout when downloading the ICU zip file. While I have manually set the timeout to 300s this does not seem to be passed to the ICU download. Thanks

R version 4.2.1 (2022-06-23) -- "Funny-Looking Kid"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> getOption("timeout")
[1] 60
> options(timeout = max(300, getOption("timeout")))
> getOption("timeout")
[1] 300
 
> install.packages('stringi',verbose=T)
Installing package into ‘xxxx/library/R-4.2/x86_64-pc-linux-gnu’
(as ‘lib’ is unspecified)
system (cmd0): /usr/local/lib/R/bin/R CMD INSTALL
trying URL 'https://cran.rstudio.com/src/contrib/stringi_1.7.8.tar.gz'
Content type 'application/x-gzip' length 8032287 bytes (7.7 MB)
==================================================
downloaded 7.7 MB

foundpkgs: stringi, /tmp/Rtmp5Wf8Jw/downloaded_packages/stringi_1.7.8.tar.gz
files: /tmp/Rtmp5Wf8Jw/downloaded_packages/stringi_1.7.8.tar.gz
Warning message:
Project requested R version '4.1.0' but '4.2.1' is currently being used 
* installing *source* package ‘stringi’ ...
** package ‘stringi’ successfully unpacked and MD5 sums checked
** using staged installation
checking for R_HOME... /usr/local/lib/R
checking for R... /usr/local/lib/R/bin/R
checking for endianness... little
checking for R >= 3.1.0 for C++11 use... yes
checking for R < 3.4.0 for CXX1X flag use... no
checking for cat... /usr/bin/cat
checking for local ICUDT_DIR... icu69/data
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking elf.h usability... yes
checking elf.h presence... yes
checking for elf.h... yes
checking whether we are using the GNU C++ compiler... yes
checking whether g++ -std=gnu++11 accepts -g... yes
checking whether the C++ compiler supports the long long type... yes
checking whether the compiler implements namespaces... yes
checking whether the compiler supports the Standard Template Library... yes
checking whether std::map is available... yes
checking for pkg-config... no
*** pkg-config cannot be found
*** Trying with 'standard' fallback flags
checking whether an ICU4C-based project can be built... no
*** This version of ICU4C cannot be used.
*** Using the ICU 69 bundle.
checking whether we may compile src/icu69/common/putil.cpp... yes
checking whether we may compile src/icu69/i18n/number_affixutils.cpp... yes
checking whether alignof(std::max_align_t) is available... yes
checking whether the ICU data library can be downloaded... downloading the ICU data library (icudt)
output path: icu69/data/icu4c-69_1-data-bin-l.zip
trying URL 'https://raw.githubusercontent.com/gagolews/stringi/master/src/icu69/data/icu4c-69_1-data-bin-l.zip'
Content type 'application/zip' length 11454999 bytes (10.9 MB)
==========
downloaded 2.3 MB

Error in download.file(paste(href, fname, sep = ""), icudtzipfname, mode = "wb"): download from 'https://raw.githubusercontent.com/gagolews/stringi/master/src/icu69/data/icu4c-69_1-data-bin-l.zip' failed

trying URL 'http://raw.githubusercontent.com/gagolews/stringi/master/src/icu69/data/icu4c-69_1-data-bin-l.zip'
Content type 'application/zip' length 11454999 bytes (10.9 MB)
===========
downloaded 2.5 MB

Error in download.file(paste(href, fname, sep = ""), icudtzipfname, mode = "wb"): download from 'http://raw.githubusercontent.com/gagolews/stringi/master/src/icu69/data/icu4c-69_1-data-bin-l.zip' failed

icudt download failed
Error: Stopping on error
In addition: Warning messages:
1: In download.file(paste(href, fname, sep = ""), icudtzipfname, mode = "wb") :
  downloaded length 2449007 != reported length 11454999
2: In download.file(paste(href, fname, sep = ""), icudtzipfname, mode = "wb") :
  URL 'https://raw.githubusercontent.com/gagolews/stringi/master/src/icu69/data/icu4c-69_1-data-bin-l.zip': Timeout of 60 seconds was reached
3: In download.file(paste(href, fname, sep = ""), icudtzipfname, mode = "wb") :
  downloaded length 2579957 != reported length 11454999
4: In download.file(paste(href, fname, sep = ""), icudtzipfname, mode = "wb") :
  URL 'https://raw.githubusercontent.com/gagolews/stringi/master/src/icu69/data/icu4c-69_1-data-bin-l.zip': Timeout of 60 seconds was reached
Execution halted
*** *********************************************************************
*** stringi cannot be built.
*** Failed to download the ICU data library (icudt). Stopping now.
*** For build environments that have no internet access,
*** see the INSTALL file for a workaround.
*** *********************************************************************
ERROR: configuration failed for package ‘stringi’
* removing ‘xxx/library/R-4.2/x86_64-pc-linux-gnu/stringi’

The downloaded source packages are in
	‘/tmp/Rtmp5Wf8Jw/downloaded_packages’
Warning message:
In utils::install.packages(pkgs = pkgs, lib = lib, repos = repos,  :
  installation of package ‘stringi’ had non-zero exit status

[gagolews]

Well, github servers are usually very fast + they are spread around the world, so either they had some temporary issues or the problem is your internet provider's side?

[MatMatt]

It's most probably on my side (newly installed dockerised Rstudio-server, where we have still to solve some connectivity issues).

But the effect of (I guess) download.file(paste(href, fname, sep = ""), icudtzipfname, mode = "wb") not being able to 'see' the global 'timeout' setting remains a potential issue. I would assume that it is an environment issue?

If you have a easy fix it would be great and make the installation more robust. But I understand that it is a issue affecting only rare cases...(I hope to soon solve my connectivity problem...).

[MatMatt]

I have solved the issue on my side by externally downloading the ICU file and doing: install.packages("stringi", configure.vars="ICUDT_DIR=xxx") as described in the INSTALL file.

[gagolews]

Another solution to this (in case anyone has a similar problem in the future) would be to consider what help("download.file") says about the timeout:

The timeout for many parts of the transfer can be set by the option timeout which defaults to 60 seconds. This is often insufficient for downloads of large files (50MB or more) and so should be increased when download.file is used in packages to do so. Note that the user can set the default timeout by the environment variable R_DEFAULT_INTERNET_TIMEOUT in recent versions of R, so to ensure that this is not decreased packages should use something likeoptions(timeout = max(300, getOption("timeout"))).

[MatMatt]

Well this is what I tried right before running install.packages

> getOption("timeout")
[1] 60
> options(timeout = max(300, getOption("timeout")))
> getOption("timeout")
[1] 300
> install.packages('stringi',verbose=T)
...

and instead of using my 300s timeout the installation process had a 60s timeout. Anyway there is a potential solution for a very rare issue...

[gagolews]

Sorry, I meant trying to set the R_DEFAULT_INTERNET_TIMEOUT environment variable, perhaps via Sys.setenv. :)

Maybe this could help fix the issue?

Marek Gagolewski https://www.gagolewski.com

Sent from my mobile device. Please excuse my brevity.

[gagolews]

Interestingly, a somewhat related issue appeared recently; see #485

[gagolews]

So maybe installing via:

R_DEFAULT_INTERNET_TIMEOUT=300 Rscript --vanilla -e 'install.packages("stringi")'

will do the trick?

[gagolews]

Or rather:

Rscript --vanilla -e 'install.packages("stringi", configure.vars="R_DEFAULT_INTERNET_TIMEOUT=300")'

[MatMatt]

Ok I thought that options(timeout = xxx) was acting on R_DEFAULT_INTERNET_TIMEOUT.

I struggle with testing because ICU is now installed, and it seems that my slow internet connection is working properly again (if you can advice me how to remove the ICU I can try).

Anyway I have tried to do within the R session: install.packages("stringi", configure.vars="R_DEFAULT_INTERNET_TIMEOUT=300") and at least there were no complaints...

The similarity in reading https://github.com/gagolews/stringi/issues/485 is that in both cases the download of the ICU seems to happen in a totally detached environment...It might be an issue that is even below the installation of 'stringi'...

[gagolews]

Thanks!

[denghb001]

I have solved the issue on my side by externally downloading the ICU file and doing: install.packages("stringi", configure.vars="ICUDT_DIR=xxx") as described in the INSTALL file.

Very good solution