Improved PTS support (presentation time stamps)
This is a large refactor of how FFmpegReader class utilizes the timestamps, to make better/quicker decisions about audio/video data.
Improvements:
- Improved PTS offset logic (to offset our streams so that at least one of them starts at position 0.0)
- Removed missing frame detection (not needed anymore)
- Use timestamps to reason about missing frames
- Fixed many async decoding issues (jumbled frames, missing beginning or ending frames)
- Much improved support for video/audio files with missing chunks/gaps (i.e. missing timestamps)
- Improved CheckWorkingFrames() method, to determine when working frames are ready to be finalized
- Improved FPS detection for invalid or missing FPS
- Fixed a crash (only reproducible on Windows), caused by 2 calls to AddImage() for the same frame
What does all this mean? Video/audio files contain packets. Packets have timestamps to help us keep them in order, and know when to use them. However, packets must be decoded into actual usable data first (compressed bytes => image/audio data). Modern decoders (FFmpeg v3+) utilize threads to decode this data, and handle things asynchronously. This refactor mainly focuses on this async code, to dramatically improve the reliability, speed, and accuracy of the frame data we are accessing with FFmpegReader. OpenShot still had lots of legacy code, which expected the current packet to be processed synchronously from the decoder, causing all sorts of mis-timing events, or flat out missing frames.
Bottom line: This PR will improve the performance for most users, prevent crashes for some users, and improve compatibility with many, many video files.
Codecov Report
Merging #837 (0419f2c) into develop (5776efd) will increase coverage by
1.04%. The diff coverage is80.15%.
@@ Coverage Diff @@
## develop #837 +/- ##
===========================================
+ Coverage 48.39% 49.43% +1.04%
===========================================
Files 184 184
Lines 17889 15780 -2109
===========================================
- Hits 8657 7801 -856
+ Misses 9232 7979 -1253
| Impacted Files | Coverage Δ | |
|---|---|---|
| src/CacheBase.h | 75.00% <ø> (ø) |
|
| src/CacheMemory.h | 0.00% <ø> (ø) |
|
| src/FFmpegReader.h | 66.66% <ø> (ø) |
|
| src/CacheDisk.cpp | 82.20% <33.33%> (-2.24%) |
:arrow_down: |
| src/Frame.cpp | 44.10% <50.00%> (+0.15%) |
:arrow_up: |
| src/CacheMemory.cpp | 87.01% <71.42%> (+0.26%) |
:arrow_up: |
| src/FFmpegReader.cpp | 73.68% <82.93%> (+7.48%) |
:arrow_up: |
| src/FrameMapper.cpp | 94.00% <100.00%> (+0.31%) |
:arrow_up: |
| src/QtUtilities.h | 80.00% <100.00%> (-3.34%) |
:arrow_down: |
| src/Timeline.cpp | 43.23% <100.00%> (-1.60%) |
:arrow_down: |
| ... and 127 more |
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more