snp_asGeneticPos kept giving the same error message

[YunfengRuan]

Hi! I need to generate the LD reference but the "snp_asGeneticPos" kept giving the same error message every time. I tried different versions of R and different input datasets, including the datasets that once gave expected results before, but the error messages are just the same. Does anyone have the similar issue?

Thank you very much for your help.

Here is the error message

*** caught segfault *** address 0x20, cause 'memory not mapped'

Traceback: 1: Module(module, mustStart = TRUE, where = env) 2: doTryCatch(return(expr), name, parentenv, handler) 3: tryCatchOne(expr, names, parentenv, handlers[[1L]]) 4: tryCatchList(expr, classes, parentenv, handlers) 5: tryCatch(Module(module, mustStart = TRUE, where = env), error = function(e) e) 6: loadModule(module, NULL, env = where, loadNow = TRUE) 7: Rcpp::loadRcppClass(Class = "WKNNF", where = NS) 8: (function (NS) Rcpp::loadRcppClass(Class = "WKNNF", where = NS))() 9: doTryCatch(return(expr), name, parentenv, handler) 10: tryCatchOne(expr, names, parentenv, handlers[[1L]]) 11: tryCatchList(expr, classes, parentenv, handlers) 12: tryCatch((function (NS) Rcpp::loadRcppClass(Class = "WKNNF", where = NS))(), error = function(e) e) 13: eval(substitute(tryCatch(FUN(WHERE), error = function(e) e), list(FUN = f, WHERE = where)), where) 14: eval(substitute(tryCatch(FUN(WHERE), error = function(e) e), list(FUN = f, WHERE = where)), where) 15: .doLoadActions(where, attach) 16: methods::cacheMetaData(ns, TRUE, ns) 17: loadNamespace(name) 18: doTryCatch(return(expr), name, parentenv, handler) 19: tryCatchOne(expr, names, parentenv, handlers[[1L]]) 20: tryCatchList(expr, classes, parentenv, handlers) 21: tryCatch(loadNamespace(name), error = function(e) stop(e)) 22: getNamespace(ns) 23: asNamespace(ns) 24: getExportedValue(pkg, name) 25: nabor::knn 26: FUN(ind = ind.part, ...) 27: eval(xpr, envir = envir) 28: eval(xpr, envir = envir) 29: doTryCatch(return(expr), name, parentenv, handler) 30: tryCatchOne(expr, names, parentenv, handlers[[1L]]) 31: tryCatchList(expr, classes, parentenv, handlers) 32: tryCatch(eval(xpr, envir = envir), error = function(e) e) 33: doTryCatch(return(expr), name, parentenv, handler) 34: tryCatchOne(expr, names, parentenv, handlers[[1L]]) 35: tryCatchList(expr, classes, parentenv, handlers) 36: tryCatch({ repeat { args <- nextElem(it) if (obj$verbose) { cat(sprintf("evaluation # %d:\n", i)) print(args) } for (a in names(args)) assign(a, args[[a]], pos = envir, inherits = FALSE) r <- tryCatch(eval(xpr, envir = envir), error = function(e) e) if (obj$verbose) { cat("result of evaluating expression:\n") print(r) } tryCatch(accumulator(list(r), i), error = function(e) { cat("error calling combine function:\n") print(e) NULL }) i <- i + 1 }}, error = function(e) { if (!identical(conditionMessage(e), "StopIteration")) stop(simpleError(conditionMessage(e), expr))}) 37: e$fun(obj, substitute(ex), parent.frame(), e$data) 38: foreach(ic = rows_along(intervals)) %dopar% { ind.part <- ind[seq(intervals[ic, "lower"], intervals[ic, "upper"])] FUN(ind = ind.part, ...)} 39: bigparallelr::split_parapply(function(ind, data, query, k, ...) { nabor::knn(data, query[ind, , drop = FALSE], k = k, ...)}, ind = bigparallelr::rows_along(query), ncores = ncores, data = data, query = query, k = k, ... = ...) 40: bigutilsr::knn_parallel(as.matrix(map.chr$V2), as.matrix(pos[ind.chr]), k = 1, ncores = 1) 41: FUN(ind.chr = ind.chr, ...) 42: eval(xpr, envir = envir) 43: eval(xpr, envir = envir) 44: doTryCatch(return(expr), name, parentenv, handler) 45: tryCatchOne(expr, names, parentenv, handlers[[1L]]) 46: tryCatchList(expr, classes, parentenv, handlers) 47: tryCatch(eval(xpr, envir = envir), error = function(e) e) 48: doTryCatch(return(expr), name, parentenv, handler) 49: tryCatchOne(expr, names, parentenv, handlers[[1L]]) 50: tryCatchList(expr, classes, parentenv, handlers) 51: tryCatch({ repeat { args <- nextElem(it) if (obj$verbose) { cat(sprintf("evaluation # %d:\n", i)) print(args) } for (a in names(args)) assign(a, args[[a]], pos = envir, inherits = FALSE) r <- tryCatch(eval(xpr, envir = envir), error = function(e) e) if (obj$verbose) { cat("result of evaluating expression:\n") print(r) } tryCatch(accumulator(list(r), i), error = function(e) { cat("error calling combine function:\n") print(e) NULL }) i <- i + 1 }}, error = function(e) { if (!identical(conditionMessage(e), "StopIteration")) stop(simpleError(conditionMessage(e), expr))}) 52: e$fun(obj, substitute(ex), parent.frame(), e$data) 53: foreach(ic = ord.chrs) %dopar% { ind.chr <- ind.chrs[[ic]] attr(ind.chr, "chr") <- as.numeric(names(ind.chrs)[ic]) FUN(ind.chr = ind.chr, ...)} 54: snp_split(infos.chr, function(ind.chr, pos, dir, rsid) { chr <- attr(ind.chr, "chr") basename <- paste0("chr", chr, if (type == "OMNI") ".OMNI" else "", ".interpolated_genetic_map") mapfile <- file.path(dir, basename) if (!file.exists(mapfile)) { url <- paste0("https://github.com/joepickrell/1000-genomes-genetic-maps/", "raw/master/", path, "/", basename, ".gz") gzfile <- paste0(mapfile, ".gz") utils::download.file(url, destfile = gzfile, quiet = TRUE) R.utils::gunzip(gzfile) } map.chr <- bigreadr::fread2(mapfile, showProgress = FALSE, nThread = 1) if (is.null(rsid)) { ind <- bigutilsr::knn_parallel(as.matrix(map.chr$V2), as.matrix(pos[ind.chr]), k = 1, ncores = 1)$nn.idx new_pos <- map.chr$V3[ind] } else { ind <- match(rsid[ind.chr], map.chr$V1) new_pos <- map.chr$V3[ind] indNA <- which(is.na(ind)) if (length(indNA) > 0) { pos.chr <- pos[ind.chr] new_pos[indNA] <- suppressWarnings(stats::spline(pos.chr, new_pos, xout = pos.chr[indNA], method = "hyman")$y) } } new_pos}, combine = "c", pos = infos.pos, dir = dir, rsid = rsid, ncores = ncores) 55: snp_asGeneticPos(CHR, POS, dir = "/stanley/huang_lab/home/yruan/ldpred2.ref/sas")

Possible actions: 1: abort (with core dump, if enabled) 2: normal R exit 3: exit R without saving workspace 4: exit R saving workspace

[privefl]

Ramdom things to try:

I would try to restart and reinstall; do you have a special installation, like with conda?

Does it happen when running sequential or parallel?

Do you have any missing value in the chromosomes and positions?

Is the download of the maps okay?

[privefl]

Can you show the call of the function, and some str() of the arguments

[YunfengRuan]

Hi Florian!

Thank you very much for your reply. I tried removing and reinstalling the bigsnpr packages in both R3.5 and R4.0, but neither of the versions works. I need to install GCC5.2 to have the program running. Previously I use R4.0 to successfully generate the LD reference

It happens when running sequential

I checked and there is no missing value in the chromosomes and positions

The script I used to test the package is

library(runonce)
library(bigsnpr)
options(bigreadr.nThread = 1)

bed_file = paste0(pref,".bed") # the prefix is a testing data only have chr22 information
genoCacheF = paste0(pref,".rds")
tempbk = paste0(pref,".bk")
corrCacheF = paste0(pre,'LDrefCaches',".rds")

print("finish RDS")} else {
RDS <- snp_readBed(bed_file) }
    
REF <- snp_attach(genoCacheF)
print("finish REF")

G <- REF$genotypes
print("finish G")

CHR <- as.integer(REF$map$chromosome)
print("finish CHR")

POS <- REF$map$physical.pos
print("finish POS")

CM <- snp_asGeneticPos(CHR, POS, dir=".") # This is where I got the error messages 
print("finish CM")

ind.chr <- which(CHR == 22)
print("finish ind.chr")

runonce::save_run(
    snp_cor(
        G, ind.col = ind.chr,
        infos.pos = CM[ind.chr], size = 3 / 1000
    ),
    file = corrCacheF )

[privefl]

Does this work?

info <- readRDS(url("https://ndownloader.figshare.com/files/25503788"))
bigsnpr::snp_asGeneticPos(info$chr, info$pos, dir = ".")

[YunfengRuan]

Thank you very much for your help. Unfortunately, this dataset gives the same error message on my server.

[privefl]

Can you see if the problem comes from a specific chromosome? E.g.

info <- readRDS(url("https://ndownloader.figshare.com/files/25503788"))
for (one_chr in 1:22) {
  print(chr)
  with(subset(info, chr == one_chr), bigsnpr::snp_asGeneticPos(chr, pos, dir = "."))
}

[YunfengRuan]

Hi Florian!

Thank you for the reply. I tried to your new code and also randomly picked any chr from my previous data that generated the right result before. I have the same error no matter what dataset nor which chr I used.

Do you think it could be my server's problem? Do I actually need to install or load other software/package to make the code running properly?

[privefl]

That's weird. Have you tried installing from both GitHub and CRAN?

[YunfengRuan]

I only dowloaded from CRAN before. That's a good point and I will try Github as well

[YunfengRuan]

I downloaded the packages form Github and it gives the same error message

[privefl]

Can you tell me which line fails when running:

info <- subset(readRDS(url("https://ndownloader.figshare.com/files/25503788")), chr == 22)
basename <- paste0("chr22.OMNI.interpolated_genetic_map")
mapfile <- file.path(".", basename)
if (!file.exists(mapfile)) {
  url <- paste0("https://github.com/joepickrell/1000-genomes-genetic-maps/", 
                "raw/master/interpolated_OMNI/", basename, ".gz")
  gzfile <- paste0(mapfile, ".gz")
  utils::download.file(url, destfile = gzfile, quiet = TRUE)
  R.utils::gunzip(gzfile)
}
stopifnot(file.size(mapfile) == 16940272)
map.chr <- bigreadr::fread2(mapfile)
ind <- bigutilsr::knn_parallel(
  as.matrix(map.chr$V2), as.matrix(info$pos), k = 1, ncores = 1)$nn.idx
new_pos <- map.chr$V3[ind]

[YunfengRuan]

Hi Florian! Thank you very much for your prompt reply. The error

 *** caught segfault ***
address 0x20, cause 'memory not mapped'

occured when I run

ind <- bigutilsr::knn_parallel(
  as.matrix(map.chr$V2), as.matrix(info$pos), k = 1, ncores = 1)$nn.idx

Thank you again for your patience and help.

[privefl]

Aaaaahh.. Can you try reinstalling {bigutilsr} then?

[YunfengRuan]

I tried that in both R3.5 and R4.0. And it gives the same error messages.

Sorry for all the trouble and thank you again for your help.

[privefl]

And if you try nabor::knn(as.matrix(map.chr$V2), as.matrix(info$pos), k = 1)?

[YunfengRuan]

Hi Florian!

Sorry for the late reply. I tried this and got the same error message.

 *** caught segfault ***
address 0x20, cause 'memory not mapped'

Traceback:
 1: Module(module, mustStart = TRUE, where = env)
 2: doTryCatch(return(expr), name, parentenv, handler)
 3: tryCatchOne(expr, names, parentenv, handlers[[1L]])
( And some very long error information )

I am not sure this is relevant. I also tried to install bigsnpr on my Mac. I have installed GCC and LLVM but when I download from CRAN, I had the error message of

clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [bigsnpr.so] Error 1
ERROR: compilation failed for package ‘bigsnpr’

And when I download from github, I had the error message of

> remotes::install_github("privefl/bigsnpr")
Downloading GitHub repo privefl/bigsnpr@HEAD
Error: Failed to install 'bigsnpr' from GitHub:
  Could not find tools necessary to compile a package
Call `pkgbuild::check_build_tools(debug = TRUE)` to diagnose the problem.

Thank you again for your help.

[privefl]

You need to install the necessary tools for compilation, and then reinstall the packages.

[YunfengRuan]

OK. Thank you for the advice. My current thougt is that since none of the debugging methods work for the server, perhaps there are some configuration problems with the server and actually I don't have any control of how the server is configured. Hopefully I can install the bigsnpr locally and solved the problem

[YunfengRuan]

Sorry for keeping troubling you. Could you give me a list of what other software or packages I need to install to have bigsnpr installed and running?

[privefl]

Yes, there are problems with how you install packages. For now, it seems to come from R package {nabor}, but it would be good if you could install from source (compile) so that you can install any version of any package. The hint was in this error message:

Could not find tools necessary to compile a package Call pkgbuild::check_build_tools(debug = TRUE) to diagnose the problem.

[privefl]

Then reinstall install.packages("nabor", type = "source") and try again.

[privefl]

Any update on this?

[YunfengRuan]

Thank you so much! I tried to reinstall nabor on my server. And now it works.