SGNify icon indicating copy to clipboard operation
SGNify copied to clipboard
verified publisher icon MPForte

Demo bug - TypeError: vitpose_inference_model() got an unexpected keyword argument 'model_mode'

Open linjiangya opened this issue 1 year ago • 2 comments

Hi,

May I ask what is the ViTPose version you were using?

Thanks

linjiangya avatar Mar 18 '24 22:03 linjiangya

I have modified:

  1. "vitpose-h-multi-coco.pth" to SGNify/data/coco17/vitpose-huge.pth
  2. model_mode=huge to model_spec="coco17-huge

It can run now, but there are now new problems with the sign_classifier.pkl:

Extracting 2D keypoints with ViTPose...
2024-03-19 00:08:06.130 | INFO     | vitpose.utils.inference:vitpose_inference_model:91 - ViTPose model loaded from data/coco17/vitpose-huge.pth
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:01<00:00,  1.28it/s]
Extracting 2D keypoints with MediaPipe for RPS...
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1710806887.833785 4109895 gl_context_egl.cc:85] Successfully initialized EGL. Major : 1 Minor: 5
I0000 00:00:1710806888.086654 4110288 gl_context.cc:357] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 535.154.05), renderer: NVIDIA A100-PCIE-40GB/PCIe/SSE2
  0%|                                                                                                                                           | 0/2 [00:00<?, ?it/s]INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  4.07it/s]
I0000 00:00:1710806888.677720 4109895 gl_context_egl.cc:85] Successfully initialized EGL. Major : 1 Minor: 5
I0000 00:00:1710806888.764450 4110341 gl_context.cc:357] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 535.154.05), renderer: NVIDIA A100-PCIE-40GB/PCIe/SSE2
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  7.22it/s]
I0000 00:00:1710806889.123218 4109895 gl_context_egl.cc:85] Successfully initialized EGL. Major : 1 Minor: 5
I0000 00:00:1710806889.242034 4110394 gl_context.cc:357] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 535.154.05), renderer: NVIDIA A100-PCIE-40GB/PCIe/SSE2
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  8.07it/s]
I0000 00:00:1710806889.573950 4109895 gl_context_egl.cc:85] Successfully initialized EGL. Major : 1 Minor: 5
I0000 00:00:1710806889.655928 4110447 gl_context.cc:357] GL version: 3.2 (OpenGL ES 3.2 NVIDIA 535.154.05), renderer: NVIDIA A100-PCIE-40GB/PCIe/SSE2
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  9.62it/s]
Segmenting signs...
Running SMPLify-X inside segmentation window for right hand...
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [02:17<00:00, 68.62s/it]
Finding sign class...
/envs/sgnify/lib/python3.10/site-packages/scipy/spatial/distance.py:647: RuntimeWarning: invalid value encountered in scalar divide
  dist = 1.0 - uv / math.sqrt(uu * vv)
/envs/sgnify/lib/python3.10/site-packages/scipy/spatial/distance.py:647: RuntimeWarning: invalid value encountered in divide
  dist = 1.0 - uv / math.sqrt(uu * vv)
Traceback (most recent call last):
  File "SGNify/sgnify.py", line 482, in <module>
    main(args)
  File "SGNify/sgnify.py", line 401, in main
    compute_sign_class(
  File "SGNify/utils/classify_sign.py", line 47, in compute_sign_class
    clf = pickle.load(file)
  File "sklearn/tree/_tree.pyx", line 865, in sklearn.tree._tree.Tree.__setstate__
  File "sklearn/tree/_tree.pyx", line 1571, in sklearn.tree._tree._check_node_ndarray
ValueError: node array from the pickle has an incompatible dtype:
- expected: {'names': ['left_child', 'right_child', 'feature', 'threshold', 'impurity', 'n_node_samples', 'weighted_n_node_samples', 'missing_go_to_left'], 'formats': ['<i8', '<i8', '<i8', '<f8', '<f8', '<i8', '<f8', 'u1'], 'offsets': [0, 8, 16, 24, 32, 40, 48, 56], 'itemsize': 64}
- got     : [('left_child', '<i8'), ('right_child', '<i8'), ('feature', '<i8'), ('threshold', '<f8'), ('impurity', '<f8'), ('n_node_samples', '<i8'), ('weighted_n_node_samples', '<f8')]

This problem was solved by changing scikit-learn version.

linjiangya avatar Mar 19 '24 01:03 linjiangya

After fixing last problem by changing scikit-learn version. THere is a new issue when going to SPECTRE:

Finding the RPS using valid MediaPipe frames...
Finding betas...
Running SPECTRE...
/envs/sgnify/lib/python3.10/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
/envs/sgnify/lib/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet50_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet50_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Downloading: "https://github.com/pytorch/vision/zipball/v0.8.1" to /homes/zy3023/.cache/torch/hub/v0.8.1.zip
/envs/sgnify/lib/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=MobileNet_V2_Weights.IMAGENET1K_V1`. You can also use `weights=MobileNet_V2_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
/envs/sgnify/lib/python3.10/site-packages/pytorch3d/io/obj_io.py:548: UserWarning: Mtl file does not exist: /code/SL/SGNify/spectre/data/template.mtl
  warnings.warn(f"Mtl file does not exist: {f}")
/envs/sgnify/lib/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=None`.
  warnings.warn(msg)
Traceback (most recent call last):
  File "/code/SL/SGNify/utils/spectre_demo.py", line 228, in <module>
    main(parser.parse_args())
  File "/code/SL/SGNify/utils/spectre_demo.py", line 84, in main
    landmarks = extract_frames(image_paths)
  File "/code/SL/SGNify/utils/spectre_demo.py", line 31, in extract_frames
    face_tracker = FaceTracker()
  File "/code/SL/SGNify/spectre/external/Visual_Speech_Recognition_for_Multiple_Languages/tracker/face_tracker.py", line 23, in __init__
    self.face_detector = RetinaFacePredictor(
  File "/code/SL/SGNify/spectre/external/face_detection/ibug/face_detection/retina_face/retina_face_predictor.py", line 28, in __init__
    pretrained_dict = torch.load(model.weights, map_location=self.device)
  File "/envs/sgnify/lib/python3.10/site-packages/torch/serialization.py", line 795, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/envs/sgnify/lib/python3.10/site-packages/torch/serialization.py", line 1002, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, 'v'.
Traceback (most recent call last):
  File "/code/SL/SGNify/sgnify.py", line 482, in <module>
    main(args)
  File "/code/SL/SGNify/sgnify.py", line 429, in main
    call_spectre(images_folder=images_folder, output_folder=spectre_folder)
  File "/code/SL/SGNify/sgnify.py", line 174, in call_spectre
    run(
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['python', 'utils/spectre_demo.py', '--images_folder', PosixPath('/code/SL/SGNify/frame_low/.tmp/images'), '--output_folder', PosixPath('/code/SL/SGNify/frame_low/.tmp/spectre')]' returned non-zero exit status 1.

After trying https://github.com/MPForte/SGNify/issues/3#issue-1765386086, it still exists:

Finding betas...
Running SPECTRE...
/envs/sgnify/lib/python3.10/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(
/envs/sgnify/lib/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet50_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet50_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Downloading: "https://github.com/pytorch/vision/zipball/main" to /homes/zy3023/.cache/torch/hub/main.zip
Traceback (most recent call last):
  File "/code/SL/SGNify/utils/spectre_demo.py", line 228, in <module>
    main(parser.parse_args())
  File "/code/SL/SGNify/utils/spectre_demo.py", line 78, in main
    spectre = SPECTRE(spectre_cfg, args.device)
  File "/code/SL/SGNify/spectre/src/spectre.py", line 38, in __init__
    self._create_model(self.cfg.model)
  File "/code/SL/SGNify/spectre/src/spectre.py", line 70, in _create_model
    self.E_expression = PerceptualEncoder(model_cfg.n_exp, model_cfg).to(self.device)
  File "/code/SL/SGNify/spectre/src/models/encoders.py", line 13, in __init__
    self.encoder = torch.hub.load('pytorch/vision', 'mobilenet_v2', pretrained=True) 
  File "/envs/sgnify/lib/python3.10/site-packages/torch/hub.py", line 542, in load
    model = _load_local(repo_or_dir, model, *args, **kwargs)
  File "/envs/sgnify/lib/python3.10/site-packages/torch/hub.py", line 569, in _load_local
    hub_module = _import_module(MODULE_HUBCONF, hubconf_path)
  File "/envs/sgnify/lib/python3.10/site-packages/torch/hub.py", line 90, in _import_module
    spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/homes/zy3023/.cache/torch/hub/pytorch_vision_main/hubconf.py", line 74, in <module>
    from torchvision.models.video import (
ImportError: cannot import name 'swin3d_b' from 'torchvision.models.video' (/envs/sgnify/lib/python3.10/site-packages/torchvision/models/video/__init__.py)
Traceback (most recent call last):
  File "/code/SL/SGNify/sgnify.py", line 482, in <module>
    main(args)
  File "/code/SL/SGNify/sgnify.py", line 429, in main
    call_spectre(images_folder=images_folder, output_folder=spectre_folder)
  File "/code/SL/SGNify/sgnify.py", line 174, in call_spectre
    run(
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['python', 'utils/spectre_demo.py', '--images_folder', PosixPath('/code/SL/SGNify/frame_low/.tmp/images'), '--output_folder', PosixPath('/code/SL/SGNify/frame_low/.tmp/spectre')]' returned non-zero exit status 1.

Can you confirm which of the torchvision version you are using?

linjiangya avatar Mar 24 '24 00:03 linjiangya

#22 All the versions are here.

However, I updated the environment and code.

MPForte avatar Oct 15 '24 16:10 MPForte