human_action question

[zixian-wu]

I've tried many times and the human_action score stays at 0. Can you give me some ideas? { "human_action": [ 0.0, [ { "video_path": "./sampled_videos/anime-new-vbench/0001.mp4", "video_results": false, "cor_num_per_video": 0 }, { "video_path": "./sampled_videos/anime-new-vbench/0002.mp4", "video_results": false, "cor_num_per_video": 0 }, { "video_path": "./sampled_videos/anime-new-vbench/0003.mp4", "video_results": false, "cor_num_per_video": 0 }, { "video_path": "./sampled_videos/anime-new-vbench/0004.mp4", "video_results": false, "cor_num_per_video": 0 }, { "video_path": "./sampled_videos/anime-new-vbench/0005.mp4", "video_results": false, "cor_num_per_video": 0 } ]

[LazySheeeeeep]

Mabey because the video_path is not correct. I was trying to evaluate some videos generated by myself and I met the same problem. Then I read the code in vbench/human_action.py, I found they may store action information in the file name. And they detect the actions in the video, compare them with the target action which appears in the file name.

[DidiD1]

i also find this question, it shows that the encode embedding of the video is 0, so strange

[JustinYuu]

Thanks for your interest in our work! For the human action dimension, since the classifier categories are fixed, we do not support customized video evaluation yet, and the name of the sampled video should be the same as the prompt for evaluation when using our provided prompt list.

[huangjch526]

so how to compute the long-prompt's human action?

[ziqihuangg]

VBench's human_action dimension currently operates only on the predefined benchmark prompts.

If you're referring to augmented prompts, you can still use the same evaluation pipeline—just apply it to the longer prompts derived from the human_action short prompts.

If you're referring to open-domain prompts, that’s not supported yet in this dimension. However, feel free to explore VBench-2.0, which offers expanded evaluation options for motion rationality and human fidelity.

Thanks!