fdetsch
getHDF cannot open connection
Hello! Apologies if this issue has already been raised elsewhere -- I didn't see it in the list.
I'm hitting an error with getHDF: b1 <- getHdf(product = "M.D11A1", begin = "2016.12.01", tileH = 18:19, tileV = 4)
cannot create dir '/var/folders/0d/qm_pqljx11s_ddc42g1_yscr0000gn/T/RtmpQjGo6F/MODIS_ARC/PROCESSED/.auxiliaries', reason 'No such file or directory'cannot create file '/var/folders/0d/qm_pqljx11s_ddc42g1_yscr0000gn/T/RtmpQjGo6F/MODIS_ARC/PROCESSED/.auxiliaries//collections.RData', reason 'No such file or directory'cannot open compressed file '/var/folders/0d/qm_pqljx11s_ddc42g1_yscr0000gn/T/RtmpQjGo6F/MODIS_ARC/PROCESSED/.auxiliaries//collections.RData', probable reason 'No such file or directory'Error in readChar(con, 5L, useBytes = TRUE) : cannot open the connection
The long string above refers to my TMPDIR folder (on Mac). After hitting this error, I've tried creating the folder structure within my TMPDIR of MODIS_ARC/PROCESSED/.auxilliaries, and also created collections.Rdata, to no avail. Thanks a lot for any advice!
No problems over here using Windows tempdir() (make sure to use the 'develop' version until the next CRAN release).
# devtools::install_github("MatMatt/MODIS", ref = "develop")
library(MODIS)
lap = file.path(tempdir(), "MODIS_ARC")
odp = file.path(lap, "PROCESSED")
MODISoptions(lap, odp)
b1 <- getHdf(product = "M.D11A1", begin = "2016.12.01", end = "2016.12.02"
, tileH = 18:19, tileV = 4)
b1
# $`MOD11A1.006`
# [1] "C:/Users/florianD/AppData/Local/Temp/RtmpU7Rre8/MODIS_ARC/MODIS/MOD11A1.006/2016.12.01/MOD11A1.A2016336.h18v04.006.2016340141659.hdf"
# [2] "C:/Users/florianD/AppData/Local/Temp/RtmpU7Rre8/MODIS_ARC/MODIS/MOD11A1.006/2016.12.01/MOD11A1.A2016336.h19v04.006.2016340140014.hdf"
# [3] "C:/Users/florianD/AppData/Local/Temp/RtmpU7Rre8/MODIS_ARC/MODIS/MOD11A1.006/2016.12.02/MOD11A1.A2016337.h18v04.006.2016340142327.hdf"
# [4] "C:/Users/florianD/AppData/Local/Temp/RtmpU7Rre8/MODIS_ARC/MODIS/MOD11A1.006/2016.12.02/MOD11A1.A2016337.h19v04.006.2016340140534.hdf"
#
# $MYD11A1.006
# [1] "C:/Users/florianD/AppData/Local/Temp/RtmpU7Rre8/MODIS_ARC/MODIS/MYD11A1.006/2016.12.01/MYD11A1.A2016336.h18v04.006.2016338081708.hdf"
# [2] "C:/Users/florianD/AppData/Local/Temp/RtmpU7Rre8/MODIS_ARC/MODIS/MYD11A1.006/2016.12.01/MYD11A1.A2016336.h19v04.006.2016338081708.hdf"
# [3] "C:/Users/florianD/AppData/Local/Temp/RtmpU7Rre8/MODIS_ARC/MODIS/MYD11A1.006/2016.12.02/MYD11A1.A2016337.h18v04.006.2016338180949.hdf"
# [4] "C:/Users/florianD/AppData/Local/Temp/RtmpU7Rre8/MODIS_ARC/MODIS/MYD11A1.006/2016.12.02/MYD11A1.A2016337.h19v04.006.2016338180947.hdf"
I have no experience with MAC, so this will be hard for me to address. MODIS's 'localArcPath' used to be created in the user's home directory by default; however, we recently had to change this to tempdir() due to a violation of CRAN policies. I assume R does have write access to tempdir()? What happens if you change 'localArcPath' and 'outDirPath' to some other folder, say your home directory?
Hi @fdetsch thanks for looking into this and sorry for the slow response! I am working on getting the MODIS reprojection tool on my Mac so I can give this a proper test.
In fact, you don't need the MRT for testing. A valid GDAL installation should be sufficient.
Hello, I am having the same or a similar issue on Windows:
# devtools::install_github("MatMatt/MODIS", ref = "develop")
library(MODIS)
> lap = file.path('C:\\temp\\', "MODIS_ARC")
> odp = file.path(lap, "PROCESSED")
> MODISoptions(lap, odp)
'MRT_HOME' not set/found! MRT is NOT enabled! See: 'https://lpdaac.usgs.gov/tools/modis_reprojection_tool'
File '~/.MODIS_Opts.R' does not exist. Create it now to make settings permanent? [y/n]: n
STORAGE:
_______________
localArcPath : C:/temp/MODIS_ARC/
outDirPath : C:/temp/MODIS_ARC/PROCESSED/
DOWNLOAD:
_______________
MODISserverOrder : LAADS, LPDAAC
dlmethod : auto
stubbornness : high
wait : 0.5
quiet : TRUE
PROCESSING:
_______________
GDAL : GDAL 1.7.0b2, FWTools 2.4.7, released 2010/01/19
MRT : Not available. Use 'MODIS:::checkTools('MRT')' for more information!
pixelSize : asIn
outProj : asIn
resamplingType : NN
dataFormat : GTiff
a <- getHdf('MOD09GQ', begin='2003219', end='2003219', extent='Austria')
Updating collection from LPDAAC for product: MOD09GQ
Error in readLines(con) :
Timeout was reached: Operation timed out after 10000 milliseconds with 0 out of 0 bytes received
Error in readLines(con) :
Timeout was reached: Operation timed out after 10000 milliseconds with 0 out of 0 bytes received
Error in readLines(con) :
Timeout was reached: Operation timed out after 10000 milliseconds with 0 out of 0 bytes received
Error in readLines(con) :
Timeout was reached: Operation timed out after 10015 milliseconds with 0 out of 0 bytes received
Error in readLines(con) :
Timeout was reached: Operation timed out after 10000 milliseconds with 0 out of 0 bytes received
Error in file(file, "rt") : cannot open the connection
Error in file(file, "rt") : cannot open the connection
> traceback()
9: readLines(con)
8: FUN(X[[i]], ...)
7: lapply(X = X, FUN = FUN, ...)
6: sapply(file.path(ftp, dirs, "/"), function(ftpdir) {
h <- curl::new_handle()
if (grepl("nsidc", ftpdir)) {
curl::handle_setopt(handle = h, httpauth = 1, userpwd = paste0(credentials()$login,
":", credentials()$password))
}
con = curl::curl(ftpdir, handle = h)
on.exit(try(close(con), silent = TRUE))
cnt = readLines(con)
close(con)
dts = regmatches(cnt, regexpr("[[:digit:]]{4}\\.[[:digit:]]{2}\\.[[:digit:]]{2}",
cnt))
return(length(dts) > 0)
})
5: getCollection(product = product, quiet = TRUE, forceCheck = TRUE)
4: unlist(getCollection(product = product, quiet = TRUE, forceCheck = TRUE))
3: .local(product, ...)
2: getHdf("MOD09GQ", begin = "2003219", end = "2003219", extent = "Austria")
1: getHdf("MOD09GQ", begin = "2003219", end = "2003219", extent = "Austria")
Tried running RStudio as administrator to circumvent possible permission problems.
Regards, Georg
@fdetsch Apologies for the long delay!
library(devtools)
install_github("MatMatt/MODIS")
library(MODIS)
lap = file.path(tempdir(), "MODIS_ARC")
odp = file.path(lap, "PROCESSED")
MODISoptions(lap, odp)
'outDirPath' does not exist and will be created in '/var/folders/0d/qm_pqljx11s_ddc42g1_yscr0000gn/T/RtmpW0ftDL/MODIS_ARC/PROCESSED/'. HDF4 driver seems to be lacking. Please install GDAL with HDF4 support. 'MRT_HOME' not set/found! MRT is NOT enabled! See: 'https://lpdaac.usgs.gov/tools/modis_reprojection_tool'
I'd appreciate any advice on getting HDF4 to be recognized. When I check gdal-config --formats in terminal, hdf4 is on the list. There must be something wrong with paths. Thanks!
Hello, I am having the same or a similar issue on Windows: lap = "D:/R_MODIS/MODIS_ARC"
odp = file.path(lap, "PROCESSED") MODISoptions(lap,odp,MODISserverOrder = "LPDAAC")
STORAGE:
localArcPath : D:/R_MODIS/MODIS_ARC outDirPath : D:/R_MODIS/MODIS_ARC/PROCESSED
DOWNLOAD:
MODISserverOrder : LPDAAC dlmethod : auto stubbornness : high wait : 0.5 quiet : TRUE
PROCESSING:
GDAL : 3.2.1 MRT : Version 4.1 (March 2011) pixelSize : asIn outProj : asIn resamplingType : NN dataFormat : GTiff cellchunk : 1
EarthdataLogin(usr = '', pwd = '*') getHdf(product = "MOD09A1", collection = "006",begin = "2021129", end = "2021153",extent = list(xmin = -180, xmax = 180, ymin = 60, ymax = 90)) Error in readLines(con) : Timeout was reached: [e4ftl01.cr.usgs.gov] Connection timed out after 10001 milliseconds
