neurodocker icon indicating copy to clipboard operation
neurodocker copied to clipboard
verified publisher icon ReproNim

freesurfer 6.0.0 minimal installation

Open zswgzx opened this issue 4 years ago • 11 comments

Hi, I'm trying to install freesurfer 6.0.0 with neurodocker 0.7.0 to perform longitudinal processing stream and wonder what its min version contains so that I can reduce the docker image size. I've searched within this repo but couldn't find anything informative feedback yet.

Neurodocker works perfectly when I did the [cross] part with fs6.0.0 was dockerized, so I'm just curious what file(s) can be safely skipped so that it won't fail to run on the cloud. Visualization related part would be done separately so all GUI/qdec/tcl etc. settings can be ignored if possible.

Any help would be much appreciated.

zswgzx avatar May 13 '21 16:05 zswgzx

hi @zswgzx - freesurfer 6.0.0-min was made using the commands listed in https://github.com/freesurfer/freesurfer/issues/70#issuecomment-316361886. It of course does not have the full functionality of freesurfer 6.0.0.

You can also try neurodocker's minification command. See https://github.com/ReproNim/neurodocker#minimize-existing-docker-image. You can minify your existing docker image for your specific commands.

kaczmarj avatar May 13 '21 16:05 kaczmarj

thanks for the quick reply @kaczmarj ! will take a look and see if I still have any question :) The minify option would help for sure~

zswgzx avatar May 13 '21 16:05 zswgzx

Looks like the cmd I used couldn't run through in the container with the trace in neurodocker minify option.

RuntimeError: Error: + printf 'NEURODOCKER (in container): ERROR: reprozip trace command exited with non-zero code. Command: /tmp/reprozip-miniconda/bin/reprozip trace -d /tmp/neurodocker-reprozip-trace --dont-identify-packages --overwrite $cmd
+ exit 1

The same cmd ran through somehow when I a) changed entrypoint as /bin/bash & b) without -d option in docker run.

$ docker run --rm -it -v $PWD:/$targetDir:rw --entrypoint /bin/bash $image
root@b14c3eded2b7: cd $targetDir/$subj;recon-all -base base -tp tp1 -tp tp2 -all -3T -sd $PWD
Subject Stamp: freesurfer-Linux-centos6_x86_64-stable-pub-v6.0.0-2beb96c
Current Stamp: freesurfer-Linux-centos6_x86_64-stable-pub-v6.0.0-2beb96c
INFO: SUBJECTS_DIR is $targetDir
Actual FREESURFER_HOME /opt/freesurfer-6.0.0-min
Linux b14c3eded2b7 4.4.0-79-generic #100-Ubuntu SMP Wed May 17 19:58:14 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
nu_correct: Command not found.
...
mri_diff: Command not found.

Should I try the full version first and then minify?

zswgzx avatar May 13 '21 18:05 zswgzx

Looks like the cmd I used couldn't run through in the container with the trace in neurodocker minify option.

Can you please paste the full error traceback? There should be some useful information in there about what went wrong.

Should I try the full version first and then minify?

Yes, sounds like a good idea. Feel free to update us here on your progress.

kaczmarj avatar May 13 '21 18:05 kaczmarj

Sure if it helps (updated: it still doesn't work for the full version). Ran on ubuntu 18.04 and its docker image as base, with ver0.7.0 for neurodocker and a recent pip installed docker python package.

screen outputs 
$ docker run --rm -itd --name fs6-min-test --security-opt=seccomp:unconfined -v $PWD:/test-data:rw mriinvivo.azurecr.io/fs6-min:21.0 
ead5e838f8aa45818230d7066817870cff4b42616e36296a3dc4b15a4e4033eb
$ cmd='cd /test-data/224241;recon-all -base base -tp tp1 -tp tp2 -all -3T -sd $PWD'
$ neurodocker-minify --container fs6-min-test --dirs-to-prune /opt --commands "$cmd" 
[NEURODOCKER 2021-05-13 13:18:08,312 INFO]: running command within container ead5e838f8aa45818230d7066817870cff4b42616e36296a3dc4b15a4e4033eb: bash /tmp/_trace.sh "cd /test-data/224241;recon-all -base base -tp tp1 -tp tp2 -all -3T -sd $PWD"
[NEURODOCKER 2021-05-13 13:18:08,370 INFO]: + REPROZIP_CONDA=/tmp/reprozip-miniconda
+ REPROZIP_TRACE_DIR=/tmp/neurodocker-reprozip-trace
+ CONDA_URL=https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
[NEURODOCKER 2021-05-13 13:18:08,370 INFO]: + NEURODOCKER_LOG_PREFIX='NEURODOCKER (in container)'
[NEURODOCKER 2021-05-13 13:18:08,370 INFO]: + '[' 1 -eq 0 ']'
[NEURODOCKER 2021-05-13 13:18:08,370 INFO]: + '[' -d /tmp/neurodocker-reprozip-trace ']'
[NEURODOCKER 2021-05-13 13:18:08,370 INFO]: + program_exists bzip2
[NEURODOCKER 2021-05-13 13:18:08,371 INFO]: + hash bzip2
[NEURODOCKER 2021-05-13 13:18:08,371 INFO]: + program_exists curl
+ hash curl
[NEURODOCKER 2021-05-13 13:18:08,371 INFO]: + '[' '!' -f /tmp/reprozip-miniconda/bin/reprozip ']'
[NEURODOCKER 2021-05-13 13:18:08,371 INFO]: + echo 'NEURODOCKER (in container): installing dedicated Miniconda and ReproZip.'
[NEURODOCKER 2021-05-13 13:18:08,371 INFO]: + install_conda_reprozip
[NEURODOCKER 2021-05-13 13:18:08,371 INFO]: NEURODOCKER (in container): installing dedicated Miniconda and ReproZip.
[NEURODOCKER 2021-05-13 13:18:08,371 INFO]: + TMP_CONDA_INSTALLER=/tmp/miniconda.sh
+ ls /tmp
[NEURODOCKER 2021-05-13 13:18:08,371 INFO]: _trace.sh
[NEURODOCKER 2021-05-13 13:18:08,371 INFO]: + curl -sSL -o /tmp/miniconda.sh https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
[NEURODOCKER 2021-05-13 13:18:09,379 INFO]: + ls /tmp
[NEURODOCKER 2021-05-13 13:18:09,380 INFO]: miniconda.sh
_trace.sh
[NEURODOCKER 2021-05-13 13:18:09,380 INFO]: + bash /tmp/miniconda.sh -b -f -p /tmp/reprozip-miniconda
[NEURODOCKER 2021-05-13 13:18:09,395 INFO]: PREFIX=/tmp/reprozip-miniconda
[NEURODOCKER 2021-05-13 13:18:09,660 INFO]: Unpacking payload ...
[NEURODOCKER 2021-05-13 13:18:14,683 INFO]: 0%|          | 0/36 [00:00<?, ?it/s]
[NEURODOCKER 2021-05-13 13:18:14,745 INFO]: Extracting : setuptools-50.3.1-py38h06a4308_1.conda:   0%|          | 0/36 [00:00<?, ?it/s]
[NEURODOCKER 2021-05-13 13:18:14,745 INFO]: Extracting : readline-8.0-h7b6447c_0.condaExtracting : _libgcc_mutex-0.1-main.conda:   6%|▌         | 2/36 [00:00<00:01, 32.62itExtracting : brotlipy-0.7.0-py38h27cfd23_1003.conda:   8%|▊         | 3/36 [00:00<00:00, 48.79it/s]
[NEURODOCKER 2021-05-13 13:18:14,745 INFO]: Extracting : ca-certificates-2020.10.14-0.conda:  11%|█         | 4/36 [00:00<00:00, 64.88it/s]
[NEURODOCKER 2021-05-13 13:18:14,745 INFO]: Extracting : pycparser-2.20-py_2.conda:  14%|█▍        | 5/36 [00:00<00:00, 80.90it/s]
[NEURODOCKER 2021-05-13 13:18:14,746 INFO]: Extracting : urllib3-1.25.11-py_0.conda:  17%|█▋        | 6/36 [00:00<00:00, 96.85it/s]
[NEURODOCKER 2021-05-13 13:18:14,746 INFO]: Extracting : ld_impl_linux-64-2.33.1-h53a641e_7.conda:  19%|█▉        | 7/36 [00:00<00:00, 112.71it/s]
[NEURODOCKER 2021-05-13 13:18:14,746 INFO]: Extracting : cffi-1.14.3-py38h261ae71_2.conda:  22%|██▏       | 8/36 [00:00<00:00, 128.49it/s]
[NEURODOCKER 2021-05-13 13:18:14,746 INFO]: Extracting : tqdm-4.51.0-pyhd3eb1b0_0.conda:  25%|██▌       | 9/36 [00:00<00:00, 144.17it/s]
[NEURODOCKER 2021-05-13 13:18:14,746 INFO]: Extracting : sqlite-3.33.0-h62c20be_0.conda:  28%|██▊       | 10/36 [00:00<00:00, 159.76it/s]
[NEURODOCKER 2021-05-13 13:18:14,751 INFO]: Extracting : cryptography-3.2.1-py38h3c74f83_1.conda:  31%|███       | 11/36 [00:00<00:00, 164.21it/s]
[NEURODOCKER 2021-05-13 13:18:14,751 INFO]: Extracting : six-1.15.0-py38h06a4308_0.conExtracting : pyopenssl-19.1.0-pyhd3eb1b0_1.conda:  36%|███▌      | 13/36 [00:00<00:00, 192.94it/s]
[NEURODOCKER 2021-05-13 13:18:14,751 INFO]: Extracting : ruamel_yaml-0.15.87-py38h7b6447c_1.conda:  39%|███▉      | 14/36 [00:00<00:00, 207.30it/s]
[NEURODOCKER 2021-05-13 13:18:14,751 INFO]: Extracting : requests-2.24.0-py_0.conda:  42%|████▏     | 15/36 [00:00<00:00, 221.54it/s]
[NEURODOCKER 2021-05-13 13:18:14,751 INFO]: Extracting : yaml-0.2.5-h7b6447c_0.conda:  44%|████▍     | 16/36 [00:00<00:00, 235.78it/s]
[NEURODOCKER 2021-05-13 13:18:14,759 INFO]: Extracting : xz-5.2.5-h7b6447c_0.conda:  47%|████▋     | 17/36 [00:00<00:00, 249.83it/s]
[NEURODOCKER 2021-05-13 13:18:14,815 INFO]: Extracting : conda-4.9.2-py38h06a4308_0.conda:  50%|█████     | 18/36 [00:00<00:00, 137.21it/s]
[NEURODOCKER 2021-05-13 13:18:14,815 INFO]: Extracting : conda-4.9.2-py38h06a4308_0.conda:  53%|█████▎    | 19/36 [00:00<00:00, 144.68it/s]
[NEURODOCKER 2021-05-13 13:18:14,869 INFO]: Extracting : pip-20.2.4-py38h06a4308_0.conda:  53%|█████▎    | 19/36 [00:00<00:00, 144.68it/s]
[NEURODOCKER 2021-05-13 13:18:14,869 INFO]: Extracting : pycosat-0.6.3-py38h7b6447c_1.conda:  56%|█████▌    | 20/36 [00:00<00:00, 144.68it/s]
[NEURODOCKER 2021-05-13 13:18:15,198 INFO]: Extracting : python-3.8.5-h7579374_1.conda:  58%|█████▊    | 21/36 [00:00<00:00, 144.68it/s]
[NEURODOCKER 2021-05-13 13:18:15,198 INFO]: Extracting : libgcc-ng-9.1.0-hdf63c60_0.conda:  61%|██████    | 22/36 [00:00<00:00, 144.68it/s]
[NEURODOCKER 2021-05-13 13:18:15,199 INFO]: Extracting : conda-package-handling-1.7.2-py38h03888b9_0.conda:  64%|██████▍   | 23/36 [00:00<00:00, 144.68it/s]
[NEURODOCKER 2021-05-13 13:18:15,199 INFO]: Extracting : conda-package-handling-1.7.2-py38h03888b9_0.conda:  67%|██████▋   | 24/36 [00:00<00:00, 35.90it/s]
[NEURODOCKER 2021-05-13 13:18:15,199 INFO]: Extracting : ncurses-6.2-he6710b0_1.conda:  67%|██████▋   | 24/36 [00:00<00:00, 35.90it/s]
[NEURODOCKER 2021-05-13 13:18:15,199 INFO]: Extracting : certifi-2020.6.20-pyhd3eb1b0_3.conda:  69%|██████▉   | 25/36 [00:00<00:00, 35.90it/s]
[NEURODOCKER 2021-05-13 13:18:15,199 INFO]: Extracting : chardet-3.0.4-py38h06a4308_1003.conda:  72%|███████▏  | 26/36 [00:00<00:00, 35.90it/s]
[NEURODOCKER 2021-05-13 13:18:15,199 INFO]: Extracting : libedit-3.1.20191231-h14c3975_1.conda:  75%|███████▌  | 27/36 [00:00<00:00, 35.90it/s]
[NEURODOCKER 2021-05-13 13:18:15,200 INFO]: Extracting : libstdcxx-ng-9.1.0-hdf63c60_0.conda:  78%|███████▊  | 28/36 [00:00<00:00, 35.90it/s]
[NEURODOCKER 2021-05-13 13:18:15,200 INFO]: Extracting : libffi-3.3-he6710b0_2.conda:  81%|████████  | 29/36 [00:00<00:00, 35.90it/s]
[NEURODOCKER 2021-05-13 13:18:15,200 INFO]: Extracting : wheel-0.35.1-pyhd3eb1b0_0.conda:  83%|████████▎ | 30/36 [00:00<00:00, 35.90it/s]
[NEURODOCKER 2021-05-13 13:18:15,200 INFO]: Extracting : idna-2.10-py_0.conda:  86%|████████▌ | 31/36 [00:00<00:00, 35.90it/s]
[NEURODOCKER 2021-05-13 13:18:15,200 INFO]: Extracting : tk-8.6.10-hbc83047_0.conda:  89%|████████▉ | 32/36 [00:00<00:00, 35.90it/s]
[NEURODOCKER 2021-05-13 13:18:15,201 INFO]: Extracting : pysocks-1.7.1-py38h06a4308_0.conda:  92%|█████████▏| 33/36 [00:00<00:00, 35.90it/s]
[NEURODOCKER 2021-05-13 13:18:15,201 INFO]: Extracting : zlib-1.2.11-h7b6447c_3.conda:  94%|█████████▍| 34/36 [00:00<00:00, 35.90it/s]
[NEURODOCKER 2021-05-13 13:18:15,201 INFO]: Extracting : openssl-1.1.1h-h7b6447c_0.conda:  97%|█████████▋| 35/36 [00:00<00:00, 35.90it/s]
[NEURODOCKER 2021-05-13 13:18:15,201 INFO]: 
[NEURODOCKER 2021-05-13 13:18:15,204 INFO]: 
[NEURODOCKER 2021-05-13 13:18:16,069 INFO]: Collecting package metadata (current_repodata.json): ...working...
[NEURODOCKER 2021-05-13 13:18:16,095 INFO]: done
[NEURODOCKER 2021-05-13 13:18:16,095 INFO]: Solving environment: ...working...
[NEURODOCKER 2021-05-13 13:18:16,283 INFO]: done
[NEURODOCKER 2021-05-13 13:18:16,303 INFO]: ## Package Plan ##

  environment location: /tmp/reprozip-miniconda

  added / updated specs:
    - _libgcc_mutex==0.1=main
    - brotlipy==0.7.0=py38h27cfd23_1003
    - ca-certificates==2020.10.14=0
    - certifi==2020.6.20=pyhd3eb1b0_3
    - cffi==1.14.3=py38h261ae71_2
    - chardet==3.0.4=py38h06a4308_1003
    - conda-package-handling==1.7.2=py38h03888b9_0
    - conda==4.9.2=py38h06a4308_0
    - cryptography==3.2.1=py38h3c74f83_1
    - idna==2.10=py_0
    - ld_impl_linux-64==2.33.1=h53a641e_7
    - libedit==3.1.20191231=h14c3975_1
    - libffi==3.3=he6710b0_2
    - libgcc-ng==9.1.0=hdf63c60_0
    - libstdcxx-ng==9.1.0=hdf63c60_0
    - ncurses==6.2=he6710b0_1
    - openssl==1.1.1h=h7b6447c_0
    - pip==20.2.4=py38h06a4308_0
    - pycosat==0.6.3=py38h7b6447c_1
    - pycparser==2.20=py_2
    - pyopenssl==19.1.0=pyhd3eb1b0_1
    - pysocks==1.7.1=py38h06a4308_0
    - python==3.8.5=h7579374_1
    - readline==8.0=h7b6447c_0
    - requests==2.24.0=py_0
    - ruamel_yaml==0.15.87=py38h7b6447c_1
    - setuptools==50.3.1=py38h06a4308_1
    - six==1.15.0=py38h06a4308_0
    - sqlite==3.33.0=h62c20be_0
    - tk==8.6.10=hbc83047_0
    - tqdm==4.51.0=pyhd3eb1b0_0
    - urllib3==1.25.11=py_0
    - wheel==0.35.1=pyhd3eb1b0_0
    - xz==5.2.5=h7b6447c_0
    - yaml==0.2.5=h7b6447c_0
    - zlib==1.2.11=h7b6447c_3


The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main
  brotlipy           pkgs/main/linux-64::brotlipy-0.7.0-py38h27cfd23_1003
  ca-certificates    pkgs/main/linux-64::ca-certificates-2020.10.14-0
  certifi            pkgs/main/noarch::certifi-2020.6.20-pyhd3eb1b0_3
  cffi               pkgs/main/linux-64::cffi-1.14.3-py38h261ae71_2
  chardet            pkgs/main/linux-64::chardet-3.0.4-py38h06a4308_1003
  conda              pkgs/main/linux-64::conda-4.9.2-py38h06a4308_0
  conda-package-han~ pkgs/main/linux-64::conda-package-handling-1.7.2-py38h03888b9_0
  cryptography       pkgs/main/linux-64::cryptography-3.2.1-py38h3c74f83_1
  idna               pkgs/main/noarch::idna-2.10-py_0
  ld_impl_linux-64   pkgs/main/linux-64::ld_impl_linux-64-2.33.1-h53a641e_7
  libedit            pkgs/main/linux-64::libedit-3.1.20191231-h14c3975_1
  libffi             pkgs/main/linux-64::libffi-3.3-he6710b0_2
  libgcc-ng          pkgs/main/linux-64::libgcc-ng-9.1.0-hdf63c60_0
  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0
  ncurses            pkgs/main/linux-64::ncurses-6.2-he6710b0_1
  openssl            pkgs/main/linux-64::openssl-1.1.1h-h7b6447c_0
  pip                pkgs/main/linux-64::pip-20.2.4-py38h06a4308_0
  pycosat            pkgs/main/linux-64::pycosat-0.6.3-py38h7b6447c_1
  pycparser          pkgs/main/noarch::pycparser-2.20-py_2
  pyopenssl          pkgs/main/noarch::pyopenssl-19.1.0-pyhd3eb1b0_1
  pysocks            pkgs/main/linux-64::pysocks-1.7.1-py38h06a4308_0
  python             pkgs/main/linux-64::python-3.8.5-h7579374_1
  readline           pkgs/main/linux-64::readline-8.0-h7b6447c_0
  requests           pkgs/main/noarch::requests-2.24.0-py_0
  ruamel_yaml        pkgs/main/linux-64::ruamel_yaml-0.15.87-py38h7b6447c_1
  setuptools         pkgs/main/linux-64::setuptools-50.3.1-py38h06a4308_1
  six                pkgs/main/linux-64::six-1.15.0-py38h06a4308_0
  sqlite             pkgs/main/linux-64::sqlite-3.33.0-h62c20be_0
  tk                 pkgs/main/linux-64::tk-8.6.10-hbc83047_0
  tqdm               pkgs/main/noarch::tqdm-4.51.0-pyhd3eb1b0_0
  urllib3            pkgs/main/noarch::urllib3-1.25.11-py_0
  wheel              pkgs/main/noarch::wheel-0.35.1-pyhd3eb1b0_0
  xz                 pkgs/main/linux-64::xz-5.2.5-h7b6447c_0
  yaml               pkgs/main/linux-64::yaml-0.2.5-h7b6447c_0
  zlib               pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3


Preparing transaction: ...working...
[NEURODOCKER 2021-05-13 13:18:16,649 INFO]: done
[NEURODOCKER 2021-05-13 13:18:16,649 INFO]: Executing transaction: ...working...
[NEURODOCKER 2021-05-13 13:18:17,576 INFO]: done
[NEURODOCKER 2021-05-13 13:18:18,091 INFO]: installation finished.
[NEURODOCKER 2021-05-13 13:18:18,092 INFO]: + rm -f /tmp/miniconda.sh
[NEURODOCKER 2021-05-13 13:18:18,109 INFO]: + /tmp/reprozip-miniconda/bin/python -m pip install --no-cache-dir reprozip
[NEURODOCKER 2021-05-13 13:18:18,458 INFO]: Collecting reprozip
[NEURODOCKER 2021-05-13 13:18:18,716 INFO]: Downloading reprozip-1.0.16-cp38-cp38-manylinux2010_x86_64.whl (749 kB)
[NEURODOCKER 2021-05-13 13:18:18,949 INFO]: Collecting PyYAML
[NEURODOCKER 2021-05-13 13:18:18,972 INFO]: Downloading PyYAML-5.4.1-cp38-cp38-manylinux1_x86_64.whl (662 kB)
[NEURODOCKER 2021-05-13 13:18:19,093 INFO]: Collecting rpaths>=0.8
[NEURODOCKER 2021-05-13 13:18:19,116 INFO]: Downloading rpaths-0.13-py2.py3-none-any.whl (12 kB)
[NEURODOCKER 2021-05-13 13:18:19,152 INFO]: Collecting distro
[NEURODOCKER 2021-05-13 13:18:19,174 INFO]: Downloading distro-1.5.0-py2.py3-none-any.whl (18 kB)
[NEURODOCKER 2021-05-13 13:18:19,184 INFO]: Requirement already satisfied: requests in /tmp/reprozip-miniconda/lib/python3.8/site-packages (from reprozip) (2.24.0)
[NEURODOCKER 2021-05-13 13:18:19,222 INFO]: Collecting usagestats>=0.3
[NEURODOCKER 2021-05-13 13:18:19,245 INFO]: Downloading usagestats-1.0-py2.py3-none-any.whl (10 kB)
[NEURODOCKER 2021-05-13 13:18:19,256 INFO]: Requirement already satisfied: certifi>=2017.4.17 in /tmp/reprozip-miniconda/lib/python3.8/site-packages (from requests->reprozip) (2020.6.20)
[NEURODOCKER 2021-05-13 13:18:19,257 INFO]: Requirement already satisfied: idna<3,>=2.5 in /tmp/reprozip-miniconda/lib/python3.8/site-packages (from requests->reprozip) (2.10)
[NEURODOCKER 2021-05-13 13:18:19,259 INFO]: Requirement already satisfied: chardet<4,>=3.0.2 in /tmp/reprozip-miniconda/lib/python3.8/site-packages (from requests->reprozip) (3.0.4)
[NEURODOCKER 2021-05-13 13:18:19,261 INFO]: Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /tmp/reprozip-miniconda/lib/python3.8/site-packages (from requests->reprozip) (1.25.11)
[NEURODOCKER 2021-05-13 13:18:19,351 INFO]: Installing collected packages: PyYAML, rpaths, distro, usagestats, reprozip
[NEURODOCKER 2021-05-13 13:18:19,934 INFO]: Successfully installed PyYAML-5.4.1 distro-1.5.0 reprozip-1.0.16 rpaths-0.13 usagestats-1.0
[NEURODOCKER 2021-05-13 13:18:20,162 INFO]: NEURODOCKER (in container): running reprozip trace command(s)
[NEURODOCKER 2021-05-13 13:18:20,162 INFO]: + echo 'NEURODOCKER (in container): running reprozip trace command(s)'
+ run_reprozip_trace 'cd /test-data/224241;recon-all -base base -tp tp1 -tp tp2 -all -3T -sd $PWD'
+ cmds=("$@")
[NEURODOCKER 2021-05-13 13:18:20,162 INFO]: + reprozip_base_cmd='/tmp/reprozip-miniconda/bin/reprozip trace -d /tmp/neurodocker-reprozip-trace --dont-identify-packages'
+ for cmd in "${cmds[@]}"
[NEURODOCKER 2021-05-13 13:18:20,162 INFO]: + '[' 'cd /test-data/224241;recon-all -base base -tp tp1 -tp tp2 -all -3T -sd $PWD' == 'cd /test-data/224241;recon-all -base base -tp tp1 -tp tp2 -all -3T -sd $PWD' ']'
+ continue_=--overwrite
[NEURODOCKER 2021-05-13 13:18:20,162 INFO]: + reprozip_cmd='/tmp/reprozip-miniconda/bin/reprozip trace -d /tmp/neurodocker-reprozip-trace --dont-identify-packages --overwrite cd /test-data/224241;recon-all -base base -tp tp1 -tp tp2 -all -3T -sd $PWD'
+ printf 'NEURODOCKER (in container): executing command: /tmp/reprozip-miniconda/bin/reprozip trace -d /tmp/neurodocker-reprozip-trace --dont-identify-packages --overwrite cd /test-data/224241;recon-all -base base -tp tp1 -tp tp2 -all -3T -sd $PWD\n'
[NEURODOCKER 2021-05-13 13:18:20,162 INFO]: NEURODOCKER (in container): executing command: /tmp/reprozip-miniconda/bin/reprozip trace -d /tmp/neurodocker-reprozip-trace --dont-identify-packages --overwrite cd /test-data/224241;recon-all -base base -tp tp1 -tp tp2 -all -3T -sd $PWD
[NEURODOCKER 2021-05-13 13:18:20,163 INFO]: + /tmp/reprozip-miniconda/bin/reprozip trace -d /tmp/neurodocker-reprozip-trace --dont-identify-packages --overwrite cd '/test-data/224241;recon-all' -base base -tp tp1 -tp tp2 -all -3T -sd '$PWD'
[NEURODOCKER 2021-05-13 13:18:20,389 INFO]: [REPROZIP] 18:18:20.388 CRITICAL: couldn't execute the target command (execvp returned): No such file or directory
[NEURODOCKER 2021-05-13 13:18:20,481 INFO]: [REPROZIP] 18:18:20.481 WARNING: Program exited with non-zero code 127
[NEURODOCKER 2021-05-13 13:18:20,532 INFO]: Uploading usage statistics is currently disabled
Please help us by providing anonymous usage statistics; you can enable this
by running:
    reprozip usage_report --enable
If you do not want to see this message again, you can run:
    reprozip usage_report --disable
Nothing will be uploaded before you opt in.
[NEURODOCKER 2021-05-13 13:18:20,533 INFO]: Configuration file written in /tmp/neurodocker-reprozip-trace/config.yml
Edit that file then run the packer -- use 'reprozip pack -h' for help
[NEURODOCKER 2021-05-13 13:18:20,561 INFO]: + printf 'NEURODOCKER (in container): ERROR: reprozip trace command exited with non-zero code. Command: /tmp/reprozip-miniconda/bin/reprozip trace -d /tmp/neurodocker-reprozip-trace --dont-identify-packages --overwrite cd /test-data/224241;recon-all -base base -tp tp1 -tp tp2 -all -3T -sd $PWD'
+ exit 1
Traceback (most recent call last):
  File "/home/shengwei/anaconda3/bin/neurodocker-minify", line 8, in <module>
    sys.exit(main())
  File "/home/shengwei/anaconda3/lib/python3.7/site-packages/neurodocker/reprozip/gentle/trace.py", line 169, in main
    trace_and_prune(container=args.container, commands=args.commands, directories_to_prune=args.dirs_to_prune)
  File "/home/shengwei/anaconda3/lib/python3.7/site-packages/neurodocker/reprozip/gentle/trace.py", line 86, in trace_and_prune
    raise RuntimeError("Error: {}".format(log))

zswgzx avatar May 13 '21 19:05 zswgzx

Have a feeling that the dependency may not be tracked as expected, @kaczmarj could you specify how to use that part? Thanks.

zswgzx avatar May 14 '21 14:05 zswgzx

The problem seems to be here

cd /test-data/224241;recon-all -base base -tp tp1 -tp tp2 -all -3T -sd $PWD

Do you see a line like that in the script you are running? It looks like it's being interpreted as

cd '/test-data/224241;recon-all' ...

which causes cd to fail, because there is no directory /test-data/224241;recon-all.

Can you put the cd and recon-all commands on separate lines? I'm not sure which script is interpreting that line as cd '/test-data/224241;recon-all'. It could be the neurodocker minification script.

kaczmarj avatar May 14 '21 16:05 kaczmarj

Now it's clear to me why all the funny messages appeared, so that if test command has multiple lines, they have to be separate. Maybe add this to the MD file? I thought I could just use a one liner trick, but that won't work. Lesson learned but I'll keep this issue open until it's fully solved.

zswgzx avatar May 14 '21 16:05 zswgzx

a simple cd command still won't work?

here are the commands:

$ docker run --rm -itd --name fstest --security-opt=seccomp:unconfined -v $PWD:/test:rw mriinvivo.azurecr.io/fs6:21.0
$ echo $cmd1
cd /test/224241
$ neurodocker-minify --container fstest --dirs-to-prune /opt --commands "$cmd1"

error messages:

[NEURODOCKER 2021-05-14 19:58:39,647 INFO]: NEURODOCKER (in container): ERROR: reprozip trace command exited with non-zero code. Command: /tmp/reprozip-miniconda/bin/reprozip trace -d /tmp/neurodocker-reprozip-trace --dont-identify-packages --overwrite cd /test/224241
Traceback (most recent call last):
  File "/home/shengwei/anaconda3/bin/neurodocker-minify", line 8, in <module>
    sys.exit(main())
  File "/home/shengwei/anaconda3/lib/python3.7/site-packages/neurodocker/reprozip/gentle/trace.py", line 169, in main
    trace_and_prune(container=args.container, commands=args.commands, directories_to_prune=args.dirs_to_prune)
  File "/home/shengwei/anaconda3/lib/python3.7/site-packages/neurodocker/reprozip/gentle/trace.py", line 86, in trace_and_prune
    raise RuntimeError("Error: {}".format(log))
RuntimeError: Error: NEURODOCKER (in container): ERROR: reprozip trace command exited with non-zero code. Command: /tmp/reprozip-miniconda/bin/reprozip trace -d /tmp/neurodocker-reprozip-trace --dont-identify-packages --overwrite cd /test/224241

zswgzx avatar May 15 '21 01:05 zswgzx

Any update @kaczmarj about this simple cd command that failed? Thanks.

zswgzx avatar May 26 '21 14:05 zswgzx

@zswgzx - thanks for the report. I had a chance to look into this, and it is because cd is not an executable. The error is

CRITICAL: couldn't execute the target command (execvp returned): No such file or directory

meaning that there is no file cd on the PATH. That's true, because cd is a shell builtin.

To get around this, it might be helpful to put all of your minification commands inside a single shell script, and then minify using that shell script. Like this

#!/usr/bin/env bash

cd /test/224241
echo "Other things here"

and then save that to a file, put it in the container (you can mount it at runtime) and run a minification command.

kaczmarj avatar May 31 '21 14:05 kaczmarj

This issue is stale because it has been open for 30 days with no activity.

github-actions[bot] avatar Sep 19 '23 01:09 github-actions[bot]

This issue was closed because it has been inactive for 14 days since being marked as stale.

github-actions[bot] avatar Dec 08 '23 01:12 github-actions[bot]