The performance of detecting shot changes is poor

[xiao-keeplearning]

Problem/Use Case

I found that the performance of detecting shot changes using PySceneDetect is not as effective as Adobe Premiere Pro. Could you please provide a parameter adjustment solution to achieve results comparable to Premiere Pro?

[Breakthrough]

If you are using high-framerate material (60+ FPS), you may want to experiment with the frame skip option. You might also want to increase the downscale factor if performance is a concern.

Are you exporting scenes to individual clips, or just using shot detection alone? Could you provide some additional detail about how you are using the project (e.g. are you using the command line program or the Python API)?

[xiao-keeplearning]

Thanks your reply. I am using the python api to process videos at 25fps. I use code for processing, which uses scene_manager.add_detector(ContentDetector()) to detect scene changes and do video clipping. Do you have any suggestions to improve performance?

[Breakthrough]

You can remove use of stats_manager to save some processing time, and you can also experiment with raising downscale here: https://www.scenedetect.com/docs/latest/api/scene_manager.html#scenedetect.scene_manager.SceneManager.downscale

Also note instead of VideoManager you should use the scenedetect.open_video function, e.g.:

from scenedetect import SceneManager, ContentDetector, open_video
video = open_video(test_video_file)
scene_manager = SceneManager()
scene_manager.add_detector(ContentDetector())
scene_manager.detect_scenes(video=video)

Do you know how fast it is versus Adobe Premiere?

[xiao-keeplearning]

Sorry, I didn't explain it clearly and you misunderstood it. I don't care about algorithm processing time, I'm more concerned about accuracy. I found that the current Python API cannot effectively detect scene changes, especially in videos where the background does not change much and the speaker's speech is edited and spliced. Adobe Premiere can basically effectively detect all scene changes.

[Breakthrough]

Using audio to detect splices is definitely something I would be interested in adding at some point. Do you have any samples you could share that exhibit poor performance?

[himacks]

I found your issue, I think. For some reason, I'm guessing your videos do not match the framerate detected by pyscenedetect, it is not a pyscenedetect issue but rather a mismatch in how you are processing your video. I had this same issue, where using ffmpeg I clipped a video and ffprobe stated the output was 48fps, however, when using pyscenedetect, the scene transitions where completely off, however, once I forced the framerate to what I suspected the true framerate to be (24) the scene transitions matched up completely.