UniAD
UniAD copied to clipboard
OpenDriveLab
AssertionError: MMCV==1.4.0 is used but incompatible. Please install mmcv>=2.0.0rc4, <2.2.0.
Please solve the MMCV version problem, thank you.
I am also considering the issue of the giteckout version.
ERROR
nohup: ignoring input
/root/anaconda3/envs/hhhhh/lib/python3.8/site-packages/torch/distributed/launch.py:178: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects `--local_rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions
warnings.warn(
WARNING:torch.distributed.run:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
*****************************************
Traceback (most recent call last):
File "./tools/test.py", line 14, in <module>
from mmdet3d.apis import single_gpu_test
File "/root/uuuuu/jjjjj/data/project/yyyyy/mmdetection3d/mmdet3d/__init__.py", line 21, in <module>
assert (mmcv_version >= digit_version(mmcv_minimum_version)
AssertionError: MMCV==1.4.0 is used but incompatible. Please install mmcv>=2.0.0rc4, <2.2.0.
Traceback (most recent call last):
File "./tools/test.py", line 14, in <module>
from mmdet3d.apis import single_gpu_test
File "/root/uuuuu/jjjjj/data/project/yyyyy/mmdetection3d/mmdet3d/__init__.py", line 21, in <module>
assert (mmcv_version >= digit_version(mmcv_minimum_version)
AssertionError: MMCV==1.4.0 is used but incompatible. Please install mmcv>=2.0.0rc4, <2.2.0.
Traceback (most recent call last):
File "./tools/test.py", line 14, in <module>
from mmdet3d.apis import single_gpu_test
File "/root/uuuuu/jjjjj/data/project/yyyyy/mmdetection3d/mmdet3d/__init__.py", line 21, in <module>
assert (mmcv_version >= digit_version(mmcv_minimum_version)
AssertionError: MMCV==1.4.0 is used but incompatible. Please install mmcv>=2.0.0rc4, <2.2.0.
Traceback (most recent call last):
File "./tools/test.py", line 14, in <module>
from mmdet3d.apis import single_gpu_test
File "/root/uuuuu/jjjjj/data/project/yyyyy/mmdetection3d/mmdet3d/__init__.py", line 21, in <module>
assert (mmcv_version >= digit_version(mmcv_minimum_version)
AssertionError: MMCV==1.4.0 is used but incompatible. Please install mmcv>=2.0.0rc4, <2.2.0.
Traceback (most recent call last):
File "./tools/test.py", line 14, in <module>
from mmdet3d.apis import single_gpu_test
File "/root/uuuuu/jjjjj/data/project/yyyyy/mmdetection3d/mmdet3d/__init__.py", line 21, in <module>
assert (mmcv_version >= digit_version(mmcv_minimum_version)
AssertionError: MMCV==1.4.0 is used but incompatible. Please install mmcv>=2.0.0rc4, <2.2.0.
Traceback (most recent call last):
File "./tools/test.py", line 14, in <module>
from mmdet3d.apis import single_gpu_test
File "/root/uuuuu/jjjjj/data/project/yyyyy/mmdetection3d/mmdet3d/__init__.py", line 21, in <module>
assert (mmcv_version >= digit_version(mmcv_minimum_version)
AssertionError: MMCV==1.4.0 is used but incompatible. Please install mmcv>=2.0.0rc4, <2.2.0.
Traceback (most recent call last):
File "./tools/test.py", line 14, in <module>
from mmdet3d.apis import single_gpu_test
File "/root/uuuuu/jjjjj/data/project/yyyyy/mmdetection3d/mmdet3d/__init__.py", line 21, in <module>
assert (mmcv_version >= digit_version(mmcv_minimum_version)
AssertionError: MMCV==1.4.0 is used but incompatible. Please install mmcv>=2.0.0rc4, <2.2.0.
Traceback (most recent call last):
File "./tools/test.py", line 14, in <module>
from mmdet3d.apis import single_gpu_test
File "/root/uuuuu/jjjjj/data/project/yyyyy/mmdetection3d/mmdet3d/__init__.py", line 21, in <module>
assert (mmcv_version >= digit_version(mmcv_minimum_version)
AssertionError: MMCV==1.4.0 is used but incompatible. Please install mmcv>=2.0.0rc4, <2.2.0.
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 507441) of binary: /root/anaconda3/envs/hhhhh/bin/python
Traceback (most recent call last):
File "/root/anaconda3/envs/hhhhh/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/root/anaconda3/envs/hhhhh/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/root/anaconda3/envs/hhhhh/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in <module>
main()
File "/root/anaconda3/envs/hhhhh/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main
launch(args)
File "/root/anaconda3/envs/hhhhh/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch
run(args)
File "/root/anaconda3/envs/hhhhh/lib/python3.8/site-packages/torch/distributed/run.py", line 710, in run
elastic_launch(
File "/root/anaconda3/envs/hhhhh/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
return launch_agent(self._config, self._entrypoint, list(args))
File "/root/anaconda3/envs/hhhhh/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 259, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
./tools/test.py FAILED
------------------------------------------------------------
Failures:
[1]:
time : 2024-03
host : uuuuu00000
rank : 1 (local_rank: 1)
exitcode : 1 (pid: 00000)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[2]:
time : 2024-03
host : uuuuu00000
rank : 2 (local_rank: 2)
exitcode : 1 (pid: 00000)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[3]:
time : 2024-03
host : uuuuu00000
rank : 3 (local_rank: 3)
exitcode : 1 (pid: 00000)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[4]:
time : 2024-03
host : uuuuu00000
rank : 4 (local_rank: 4)
exitcode : 1 (pid: 507449)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[5]:
time : 2024-03
host : uuuuu00000
rank : 5 (local_rank: 5)
exitcode : 1 (pid: 00000)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[6]:
time : 2024-03
host : uuuuu00000
rank : 6 (local_rank: 6)
exitcode : 1 (pid: 00000)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[7]:
time : 2024-03
host : uuuuu00000
rank : 7 (local_rank: 7)
exitcode : 1 (pid: 00000)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
time : 2024-03
host : uuuuu00000
rank : 0 (local_rank: 0)
exitcode : 1 (pid: 00000)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================
###############################################################
Ubuntu Ver: 20.04 / CUDA 11.3 / cudatoolkit=11.3 / Torch=1.10.1 / Python 3.8.19
###############################################################
Annotation processing for error resolution
.. /mmdetection3d/requirements/runtime.txt
lyft_dataset_sdk
# networkx>=2.5
numba # you should install numba==0.53.0 if your environment is cuda-9.0
# numpy
nuscenes-devkit
open3d
plyfile
scikit-image
# by default we also use tensorboard to log results
tensorboard
trimesh
###############################################################
Process
env setting -- hhhhh
conda create -n hhhhh python=3.8 -y
conda env list
conda activate hhhhh
conda list
cd root/uuuuu/jjjjj/data/project/yyyyy ctrl+shift+p : >select interpreter : hhhhh
Torch install - cudatoolkit=11.3 / Torch=1.10.1
conda install pytorch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1 cudatoolkit=11.3 -c pytorch -c conda-forge -y
conda list cudatoolkit -f
python -c "import torch; print(torch.__version__)"
export PATH=/usr/local/cuda-11.3/bin:$PATH
export CUDA_HOME=/mnt/cuda-11.3/
install mmcv
pip install mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.0/index.html
pip show mmcv-full
install mmdet, mmseg
pip install mmdet==2.14.0
pip show mmdet
pip install mmsegmentation==0.14.1
pip show mmsegmentation
install mmdet3d
git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
git checkout v1.4.0 #git:Refresh
pip install scipy==1.7.3
pip show scipy
/root/uuuuu/jjjjj/data/project/yyyyy/mmdetection3d/requirements/runtime.txt
pip install scikit-image==0.20.0
pip uninstall networkx
pip show scikit-image
pip install -v -e .
install UniAD
git clone https://github.com/OpenDriveLab/UniAD.git
cd UniAD
pip install -r requirements.txt
pip install numpy==1.24.3
pip install numpy==1.20.0
pip show numpy
install pretrained weights
mkdir ckpts && cd ckpts
wget https://github.com/zhiqi-li/storage/releases/download/v1.0/bevformer_r101_dcn_24ep.pth
wget https://github.com/OpenDriveLab/UniAD/releases/download/v1.0/uniad_base_track_map.pth
wget https://github.com/OpenDriveLab/UniAD/releases/download/v1.0.1/uniad_base_e2e.pth
cd ..
nuScenes data
cd UniAD
mkdir data && cd data
ln -s /root/uuuuu/jjjjj/data/dataset/nuscenes /root/uuuuu/jjjjj/data/project/yyyyy/UniAD/data #DATA Symbolic Link
mkdir infos && cd infos
wget https://github.com/OpenDriveLab/UniAD/releases/download/v1.0/nuscenes_infos_temporal_train.pkl
wget https://github.com/OpenDriveLab/UniAD/releases/download/v1.0/nuscenes_infos_temporal_val.pkl
cd UniAD/data
cd ..
mkdir others && cd others
wget https://github.com/OpenDriveLab/UniAD/releases/download/v1.0/motion_anchor_infos_mode6.pkl
Evaluation Example
cd UniAD
nohup ./tools/uniad_dist_eval.sh ./projects/configs/stage1_track_map/base_track_map.py ./ckpts/uniad_base_track_map.pth 8 > output.log 2>&1 &
nohup ./tools/uniad_dist_eval.sh ./projects/configs/stage2_e2e/base_e2e.py ./ckpts/uniad_base_e2e.pth 8 > output.log 2>&1 &
###############################################################
I understand that Ubuntu is installed in the 20.04 version.
python -c "from mmcv.utils import collect_env; print(collect_env())"
/bin/sh: 1: /usr/local/cuda/bin/nvcc: not found
{'sys.platform':
'linux',
'Python': '3.8.19 (default, Mar 20 2024, 19:58:24) [GCC 11.2.0]',
'CUDA available': True,
'GPU 0': 'NVIDIA A100-SXM4-80GB MIG 3g.40gb',
'CUDA_HOME': '/usr/local/cuda',
'NVCC': '',
'GCC': 'gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0',
'PyTorch': '1.10.1',
'PyTorch compiling details': 'PyTorch built with:\n - GCC 7.3\n - C++ Version: 201402\n - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications\n - Intel(R) MKL-DNN v2.2.3 (Git Hash 7336ca9f055cf1bfa13efb658fe15dc9b41f0740)\n - OpenMP 201511 (a.k.a. OpenMP 4.5)\n - LAPACK is enabled (usually provided by MKL)\n - NNPACK is enabled\n - CPU capability usage: AVX2\n - CUDA Runtime 11.3\n - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37\n - CuDNN 8.2\n - Magma 2.5.2\n - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release,
CUDA_VERSION=11.3,
CUDNN_VERSION=8.2.0,
CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++,
CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow,
LAPACK_INFO=mkl,
PERF_WITH_AVX=1,
PERF_WITH_AVX2=1,
PERF_WITH_AVX512=1,
TORCH_VERSION=1.10.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, \n', 'TorchVision': '0.11.2',
'OpenCV': '4.9.0',
'MMCV': '1.4.0',
'MMCV Compiler': 'GCC 7.3',
'MMCV CUDA Compiler': '11.3'}
###############################################################
pip list
Package Version Editable project location
------------------------- ----------- ------------------------------------------------
absl-py 2.1.0
addict 2.4.0
aiohttp 3.9.3
aiosignal 1.3.1
asttokens 2.4.1
async-timeout 4.0.3
attrs 23.2.0
backcall 0.2.0
black 24.3.0
blinker 1.7.0
cachetools 5.3.3
casadi 3.5.5
certifi 2024.2.2
charset-normalizer 3.3.2
click 8.1.7
comm 0.2.2
ConfigArgParse 1.7
contourpy 1.1.1
cycler 0.12.1
dash 2.16.1
dash-core-components 2.0.0
dash-html-components 2.0.0
dash-table 5.0.0
decorator 5.1.1
descartes 1.1.0
einops 0.4.1
exceptiongroup 1.2.0
executing 2.0.1
fastjsonschema 2.19.1
fire 0.6.0
flake8 7.0.0
Flask 3.0.2
fonttools 4.50.0
frozenlist 1.4.1
fsspec 2024.3.1
future 1.0.0
google-api-core 2.18.0
google-auth 2.29.0
google-auth-oauthlib 1.0.0
google-cloud-bigquery 3.19.0
google-cloud-core 2.4.1
google-crc32c 1.5.0
google-resumable-media 2.7.0
googleapis-common-protos 1.63.0
grpcio 1.62.1
grpcio-status 1.62.1
idna 3.6
imageio 2.34.0
importlib_metadata 7.1.0
importlib_resources 6.4.0
iniconfig 2.0.0
ipython 8.12.3
ipywidgets 8.1.2
itsdangerous 2.1.2
jedi 0.19.1
Jinja2 3.1.3
joblib 1.3.2
jsonschema 4.21.1
jsonschema-specifications 2023.12.1
jupyter_core 5.7.2
jupyterlab_widgets 3.0.10
kiwisolver 1.4.5
lazy_loader 0.3
lightning-utilities 0.11.0
llvmlite 0.41.1
lyft-dataset-sdk 0.0.8
Markdown 3.6
markdown-it-py 3.0.0
MarkupSafe 2.1.5
matplotlib 3.5.3
matplotlib-inline 0.1.6
mccabe 0.7.0
mdurl 0.1.2
mkl-fft 1.3.1
mkl-random 1.2.2
mkl-service 2.4.0
mmcv-full 1.4.0
mmdet 2.14.0
mmdet3d 1.4.0 /root/uuuuu/jjjjj/data/project/yyyyy/mmdetection3d
mmengine 0.10.3
mmsegmentation 0.14.1
motmetrics 1.1.3
multidict 6.0.5
mypy-extensions 1.0.0
nbformat 5.10.3
nest-asyncio 1.6.0
networkx 3.1
numba 0.58.1
numpy 1.24.3
nuscenes-devkit 1.1.11
oauthlib 3.2.2
olefile 0.47
open3d 0.18.0
opencv-python 4.9.0.80
packaging 24.0
pandas 2.0.3
parso 0.8.3
pathspec 0.12.1
pexpect 4.9.0
pickleshare 0.7.5
pillow 10.2.0
pip 23.3.1
pkgutil_resolve_name 1.3.10
platformdirs 4.2.0
plotly 5.20.0
pluggy 1.4.0
plyfile 1.0.3
prettytable 3.10.0
prompt-toolkit 3.0.43
proto-plus 1.23.0
protobuf 4.25.3
ptyprocess 0.7.0
pure-eval 0.2.2
pyasn1 0.5.1
pyasn1-modules 0.3.0
pycocotools 2.0.7
pycodestyle 2.11.1
pyflakes 3.2.0
Pygments 2.17.2
pyparsing 3.1.2
pyquaternion 0.9.9
pytest 8.1.1
python-dateutil 2.9.0.post0
pytorch-lightning 1.2.5
pytz 2024.1
PyWavelets 1.4.1
PyYAML 6.0.1
referencing 0.34.0
requests 2.31.0
requests-oauthlib 1.4.0
retrying 1.3.4
rich 13.7.1
rpds-py 0.18.0
rsa 4.9
scikit-image 0.20.0
scikit-learn 1.3.2
scipy 1.9.1
setuptools 68.2.2
Shapely 1.8.5.post1
six 1.16.0
stack-data 0.6.3
tenacity 8.2.3
tensorboard 2.14.0
tensorboard-data-server 0.7.2
termcolor 2.4.0
terminaltables 3.1.10
threadpoolctl 3.4.0
tifffile 2023.7.10
tomli 2.0.1
torch 1.10.1
torchaudio 0.10.1
torchmetrics 1.3.2
torchvision 0.11.2
tqdm 4.66.2
traitlets 5.14.2
trimesh 4.2.0
typing_extensions 4.10.0
tzdata 2024.1
urllib3 2.2.1
wcwidth 0.2.13
Werkzeug 3.0.1
wheel 0.41.2
widgetsnbextension 4.0.10
yapf 0.40.2
yarl 1.9.4
zipp 3.18.1
###############################################################
Please solve the MMCV version problem, thank you.
I am also considering the issue of the giteckout version.
Problem Cause: The git checkout version of mmdet3d was not matched. Thought I had to match the gitcheckout version for CUDA 11.3 and Torch 1.10.1. But it was related to mmcv. Workaround: I set the git checkout version to 0.17.1, rerouted it, and reinstalled the files.
cd ~
git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
git checkout v0.17.1
pip install scipy==1.7.3
pip install scikit-image==0.20.0
pip install -v -e .
I also processed the code below in the "../mmdetection3d/requirements/runtime.txt" path.
#networkx>=2.2,<2.3
#numpy<1.20.0
In subsequent execution, it is estimated that there will be a problem with the input data. Estimated to add a few lines of code to the path. .. /UniAD/projects/stage1_track_map/base_track_map.py".