YuLab-SMU
enrichWP mac memory issue
> library(clusterProfiler)
> data(geneList, package="DOSE")
> gene <- names(geneList)[abs(geneList) > 2]
> enrichWP(gene, organism = "Homo sapiens")
Error: vector memory limit of 64.0 Gb reached, see mem.maxVSize()
The above works perfectly in a linux machine. Its the enrichWP that is having this issue, the enrichKEGG works without any issue in the same machine, with the same gene list.
Sometime enrichWP, for the same above command, gives errors like below;
Error in simplifyPieces(pieces, n, fill == "left") :
'names' attribute [4537016448] must be the same length as the vector [3]
Below is the sessioninfo;
> sessionInfo()
R version 4.4.3 (2025-02-28) Platform: aarch64-apple-darwin20 Running under: macOS Ventura 13.7.4
Matrix products: default BLAS: /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/lib/libRblas.0.dylib LAPACK: /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/lib/libRlapack.dylib; LAPACK version 3.12.0
locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
time zone: America/New_York tzcode source: internal
attached base packages: [1] stats graphics grDevices utils datasets methods base
other attached packages: [1] clusterProfiler_4.14.6
loaded via a namespace (and not attached):
[1] tidyselect_1.2.1 dplyr_1.1.4 farver_2.1.2
[4] blob_1.2.4 R.utils_2.13.0 Biostrings_2.74.1
[7] lazyeval_0.2.2 fastmap_1.2.0 digest_0.6.37
[10] lifecycle_1.0.4 KEGGREST_1.46.0 tidytree_0.4.6
[13] RSQLite_2.3.9 magrittr_2.0.3 compiler_4.4.3
[16] rlang_1.1.5 tools_4.4.3 igraph_2.1.4
[19] data.table_1.17.0 ggtangle_0.0.6 bit_4.6.0
[22] gson_0.1.0 plyr_1.8.9 RColorBrewer_1.1-3
[25] aplot_0.2.5 BiocParallel_1.40.0 withr_3.0.2
[28] purrr_1.0.4 BiocGenerics_0.52.0 R.oo_1.27.0
[31] grid_4.4.3 stats4_4.4.3 GOSemSim_2.32.0
[34] enrichplot_1.26.6 colorspace_2.1-1 GO.db_3.20.0
[37] ggplot2_3.5.1 scales_1.3.0 cli_3.6.4
[40] crayon_1.5.3 treeio_1.30.0 generics_0.1.3
[43] ggtree_3.14.0 httr_1.4.7 reshape2_1.4.4
[46] DBI_1.2.3 qvalue_2.38.0 ape_5.8-1
[49] cachem_1.1.0 DOSE_4.0.0 stringr_1.5.1
[52] zlibbioc_1.52.0 splines_4.4.3 parallel_4.4.3
[55] ggplotify_0.1.2 AnnotationDbi_1.68.0 XVector_0.46.0
[58] yulab.utils_0.2.0 vctrs_0.6.5 Matrix_1.7-3
[61] jsonlite_1.9.1 gridGraphics_0.5-1 IRanges_2.40.1
[64] patchwork_1.3.0 S4Vectors_0.44.0 bit64_4.6.0-1
[67] ggrepel_0.9.6 tidyr_1.3.1 glue_1.8.0
[70] codetools_0.2-20 cowplot_1.1.3 stringi_1.8.4
[73] gtable_0.3.6 GenomeInfoDb_1.42.3 UCSC.utils_1.2.0
[76] munsell_0.5.1 tibble_3.2.1 pillar_1.10.1
[79] fgsea_1.32.2 GenomeInfoDbData_1.2.13 R6_2.6.1
[82] lattice_0.22-6 Biobase_2.66.0 R.methodsS3_1.8.2
[85] png_0.1-8 memoise_2.0.1 ggfun_0.1.8
[88] Rcpp_1.0.14 fastmatch_1.1-6 nlme_3.1-167
[91] fs_1.6.5 pkgconfig_2.0.3
Below is the sessioninfo from my perfectly running enrichWP linux run;
> sessionInfo()
R version 4.4.3 (2025-02-28) Platform: x86_64-pc-linux-gnu Running under: Rocky Linux 8.7 (Green Obsidian)
Matrix products: default BLAS/LAPACK: /usr/local/intel/2022.1.2.146/mkl/2022.0.2/lib/intel64/libmkl_rt.so.2; LAPACK version 3.9.0
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
time zone: America/New_York tzcode source: system (glibc)
attached base packages: [1] stats graphics grDevices utils datasets methods base
other attached packages: [1] clusterProfiler_4.14.6
loaded via a namespace (and not attached):
[1] tidyselect_1.2.1 dplyr_1.1.4 farver_2.1.2
[4] blob_1.2.4 R.utils_2.13.0 Biostrings_2.74.1
[7] lazyeval_0.2.2 fastmap_1.2.0 digest_0.6.37
[10] lifecycle_1.0.4 KEGGREST_1.46.0 tidytree_0.4.6
[13] RSQLite_2.3.9 magrittr_2.0.3 compiler_4.4.3
[16] rlang_1.1.5 tools_4.4.3 igraph_2.0.3
[19] data.table_1.17.0 ggtangle_0.0.6 bit_4.5.0.1
[22] gson_0.1.0 plyr_1.8.9 RColorBrewer_1.1-3
[25] aplot_0.2.5 BiocParallel_1.40.0 withr_3.0.2
[28] purrr_1.0.4 BiocGenerics_0.52.0 R.oo_1.27.0
[31] grid_4.4.3 stats4_4.4.3 GOSemSim_2.32.0
[34] enrichplot_1.26.6 colorspace_2.1-1 GO.db_3.20.0
[37] ggplot2_3.5.1 scales_1.3.0 cli_3.6.4
[40] crayon_1.5.3 treeio_1.30.0 generics_0.1.3
[43] ggtree_3.14.0 httr_1.4.7 reshape2_1.4.4
[46] DBI_1.2.3 qvalue_2.38.0 ape_5.8-1
[49] cachem_1.1.0 DOSE_4.0.0 stringr_1.5.1
[52] zlibbioc_1.52.0 splines_4.4.3 parallel_4.4.3
[55] ggplotify_0.1.2 AnnotationDbi_1.68.0 XVector_0.46.0
[58] yulab.utils_0.2.0 vctrs_0.6.5 Matrix_1.7-1
[61] jsonlite_1.9.1 gridGraphics_0.5-1 IRanges_2.40.1
[64] patchwork_1.3.0 S4Vectors_0.44.0 bit64_4.6.0-1
[67] ggrepel_0.9.6 tidyr_1.3.1 glue_1.8.0
[70] codetools_0.2-20 cowplot_1.1.3 stringi_1.8.4
[73] gtable_0.3.6 GenomeInfoDb_1.42.3 UCSC.utils_1.2.0
[76] munsell_0.5.1 tibble_3.2.1 pillar_1.10.1
[79] fgsea_1.32.2 GenomeInfoDbData_1.2.13 R6_2.6.1
[82] lattice_0.22-6 Biobase_2.66.0 R.methodsS3_1.8.2
[85] png_0.1-8 memoise_2.0.1 ggfun_0.1.8
[88] Rcpp_1.0.14 fastmatch_1.1-6 nlme_3.1-166
[91] fs_1.6.5 pkgconfig_2.0.3
enrichWP worked perfectly in my mac until a couple of days ago. I tried restarting R, restarting the machine, increasing the memory, but in vain.
Any thoughts/comments would be much appreciated.
I have the same problem, my package use all function from clusterProfiler since 3 years, and this problem is brand new,
Any suggestions on how to fix it, please?
Thanks in advance for any help,
Best
Simon
@boutrys I tried the renv route and it worked.
# mkdir clusterProfiler_test
# cd clusterProfiler_test
# Start R (from within this directory)
install.packages("renv")
renv::init()
# Restart R
install.packages("BiocManager")
BiocManager::install("clusterProfiler")
library(clusterProfiler)
data(geneList, package="DOSE")
gene <- names(geneList)[abs(geneList) > 2]
enrichWP(gene, organism = "Homo sapiens")
Hope this helps. Cheers, Vijay
I tried, indeed this will work
library(clusterProfiler) data(geneList, package="DOSE") gene <- names(geneList)[abs(geneList) > 2] enrichWP(gene, organism = "Homo sapiens")
But I still have a problem (that didn't existed before)
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_small_public_GVCF.R:59:3'): small test on public dataset on the GVCF ──
Error in simplifyPieces(pieces, n, fill == "left"): long vectors not supported yet: /Volumes/Builds/R4/R-4.3.1/src/include/Rinlinedfuns.h:537
Backtrace:
▆
- └─DoAggregate::main(...) at test_small_public_GVCF.R:59:3
- └─DoAggregate::over_representation_analysis(...)
-
└─clusterProfiler::enrichWP(...) -
└─clusterProfiler:::prepare_WP_data(organism) -
└─clusterProfiler:::get_wp_data(organism) -
└─gson::read.gmt.wp(url, output = output) -
├─tidyr::separate(...) -
└─tidyr:::separate.data.frame(...) -
└─tidyr:::str_separate(...) -
└─tidyr:::str_split_fixed(...) -
└─tidyr:::simplifyPieces(pieces, n, fill == "left")
[ FAIL 1 | WARN 20 | SKIP 0 | PASS 0 ] Error: Test failures Execution halted
When I just remove clusterProfiler::enrichWP from my function, then everything works
I recently encountered the same problem that I had never experienced before, which is also new to me:
wp_ls <- lapply(setNames(names(chimera_deg_filtered_ls), names(chimera_deg_filtered_ls)), function(x) {
wp = enrichWP(
gene = chimera_deg_filtered_ls[[x]]$ENTREZID,
organism = "Homo sapiens" )
write.csv(filter(wp@result, pvalue < 0.05), file = paste0("./5_DEGs/3_cluster_group_comp/chimera_sct_wilcox/wiki_pathways_enrichment/", x, "_wp.csv"))
wp
})
> Error in simplifyPieces(pieces, n, fill == "left") :
long vectors not supported yet: /Volumes/Builds/R4/R-4.3.1/src/include/Rinlinedfuns.h:537
I got another error message initially: Error: vector memory exhausted (limit reached?) But I was unable to reproduce it. It’s possible that this problem is related to a recent update of several packages in my R environment. But I’m not certain which package is causing this problem.
I also tried mclapply() to parallel the process, but it gave either error messages or caused my laptop to freeze and require a hard reboot:
library(parallel)
wp_ls <- mclapply(setNames(names(chimera_deg_filtered_ls), names(chimera_deg_filtered_ls)), function(x) {
wp = enrichWP(
gene = chimera_deg_filtered_ls[[x]]$ENTREZID,
organism = "Homo sapiens" )
write.csv(filter(wp@result, pvalue < 0.05), file = paste0("./5_DEGs/3_cluster_group_comp/chimera_sct_wilcox/wiki_pathways_enrichment/", x, "_wp.csv"))
wp
}, mc.cores = detectCores())
The laptop I'm using is an Apple Macbook Air w/ M2 chip (1TB/24GB), below is my session info:
R version 4.3.1 (2023-06-16)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS 15.3.2
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib; LAPACK version 3.11.0
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
time zone: America/New_York
tzcode source: internal
attached base packages:
[1] stats4 stats graphics grDevices utils datasets methods base
other attached packages:
[1] qs_0.27.3 lubridate_1.9.4 forcats_1.0.0 stringr_1.5.1 dplyr_1.1.4
[6] purrr_1.0.4 readr_2.1.5 tidyr_1.3.1 tibble_3.2.1 ggplot2_3.5.2
[11] tidyverse_2.0.0 org.Hs.eg.db_3.18.0 AnnotationDbi_1.64.1 IRanges_2.36.0 S4Vectors_0.40.2
[16] Biobase_2.62.0 BiocGenerics_0.48.1 clusterProfiler_4.10.1
loaded via a namespace (and not attached):
[1] DBI_1.2.3 bitops_1.0-9 shadowtext_0.1.4 gson_0.1.0
[5] gridExtra_2.3 rlang_1.1.6 magrittr_2.0.3 DOSE_3.28.2
[9] compiler_4.3.1 RSQLite_2.3.9 png_0.1-8 vctrs_0.6.5
[13] reshape2_1.4.4 pkgconfig_2.0.3 crayon_1.5.3 fastmap_1.2.0
[17] XVector_0.42.0 ggraph_2.2.1 HDO.db_0.99.1 tzdb_0.5.0
[21] enrichplot_1.22.0 bit_4.6.0 zlibbioc_1.48.2 cachem_1.1.0
[25] aplot_0.2.5 GenomeInfoDb_1.38.8 jsonlite_2.0.0 blob_1.2.4
[29] BiocParallel_1.36.0 tweenr_2.0.3 parallel_4.3.1 R6_2.6.1
[33] stringi_1.8.7 RColorBrewer_1.1-3 GOSemSim_2.28.1 Rcpp_1.0.14
[37] timechange_0.3.0 Matrix_1.6-5 splines_4.3.1 igraph_2.1.4
[41] tidyselect_1.2.1 qvalue_2.34.0 rstudioapi_0.17.1 viridis_0.6.5
[45] stringfish_0.16.0 codetools_0.2-20 lattice_0.22-7 plyr_1.8.9
[49] treeio_1.26.0 withr_3.0.2 KEGGREST_1.42.0 gridGraphics_0.5-1
[53] RcppParallel_5.1.10 scatterpie_0.2.4 polyclip_1.10-7 Biostrings_2.70.3
[57] pillar_1.10.2 ggtree_3.10.1 ggfun_0.1.8 generics_0.1.3
[61] RCurl_1.98-1.17 hms_1.1.3 munsell_0.5.1 scales_1.3.0
[65] tidytree_0.4.6 RApiSerialize_0.1.4 glue_1.8.0 lazyeval_0.2.2
[69] tools_4.3.1 data.table_1.17.0 fgsea_1.28.0 fs_1.6.6
[73] graphlayouts_1.2.2 fastmatch_1.1-6 tidygraph_1.3.1 cowplot_1.1.3
[77] grid_4.3.1 ape_5.8-1 colorspace_2.1-1 nlme_3.1-168
[81] GenomeInfoDbData_1.2.11 patchwork_1.3.0 ggforce_0.4.2 cli_3.6.4
[85] viridisLite_0.4.2 gtable_0.3.6 yulab.utils_0.2.0 digest_0.6.37
[89] ggrepel_0.9.6 ggplotify_0.1.2 farver_2.1.2 memoise_2.0.1
[93] lifecycle_1.0.4 httr_1.4.7 GO.db_3.18.0 bit64_4.6.0-1
[97] MASS_7.3-60.0.1
