InternVideo icon indicating copy to clipboard operation
InternVideo copied to clipboard
verified publisher icon OpenGVLab

NEED HELP: Action Classification low performance

Open Major1994 opened this issue 1 year ago • 4 comments

Problems: Use demo to test action classification on kinetics-700 validation set but get very poor result

Experiment:

  1. Pretrained model: https://huggingface.co/OpenGVLab/InternVideo2-Stage2_1B-224p-f4/tree/main
  2. text candidate：use the class name of k700 dataset annotation
  3. dataset：kinetics-700 validation set.
  4. code：demo.ipynb

example： input：carving ice/nTnAoTQ41Nc_000011_000021.mp4 output： text: coloring in ~ prob: 0.0085 text: acting in play ~ prob: 0.0053 text: smashing ~ prob: 0.0043 text: cracking knuckles ~ prob: 0.0041 text: tasting food ~ prob: 0.0040

Major1994 avatar May 17 '24 04:05 Major1994

Can you try the VideoCLIP model?

Andy1621 avatar May 23 '24 07:05 Andy1621

Can you try the VideoCLIP model?

Do you mean InternVideo2_CLIP in InternVideo2/multi_modality/scripts/evaluation/clip/zero_shot/1B/config_k400.py ？ I'm a green hand in this field :PP

Major1994 avatar May 23 '24 07:05 Major1994

Can you try the VideoCLIP model?

Do you mean abandon internvideo2 and turn to videoclip for action recognition？Your work seems very promising.

Major1994 avatar May 23 '24 08:05 Major1994

Yes. I'm not sure whether there is a bug in the demo. But I have tested the VideoCLIP on Kinetics and it runs normally.

Andy1621 avatar May 23 '24 08:05 Andy1621