PaulleDemon
Audio added
Added audio feature, please test it with different videos.
@PaulleDemon #2
What to test?
- Different FPS videos with audio
- Play/Pause/Seek
- Monitor the memory changes
I have separated both methods, if we use audio=False it uses the old method to read frames. The audio method uses buffer storing.
@PaulleDemon Let me explain the changes, if you can improve it.
When we decode audio and video simultaneously with container.decode(video=0, audio=0), it generally extracts frames like this:
video_frame at 0.0
video_frame at 0.04
video_frame at 0.08
video_frame at 0.13
video_frame at 0.17
video_frame at 0.21
audio_frame at 0.0
audio_frame at 0.02
audio_frame at 0.05
audio_frame at 0.07
audio_frame at 0.09
audio_frame at 0.12
audio_frame at 0.14
audio_frame at 0.16
audio_frame at 0.19
audio_frame at 0.21
audio_frame at 0.23
audio_frame at 0.26
audio_frame at 0.28
audio_frame at 0.3
audio_frame at 0.33
audio_frame at 0.35
audio_frame at 0.37
audio_frame at 0.39
audio_frame at 0.42
audio_frame at 0.44
audio_frame at 0.46
video_frame at 0.25
video_frame at 0.29
video_frame at 0.33
video_frame at 0.38
video_frame at 0.42
video_frame at 0.46
video_frame at 0.5
video_frame at 0.54
video_frame at 0.58
video_frame at 0.63
video_frame at 0.67
...
If we play both these frames in this order, then they don't sync properly.
But if we sort a small set of all these frames (one by one) according to their presentation time, we can directly play them without any delay as the audio is playing at the correct samplerate.
Sorted like this:
video_frame at 0.0
audio_frame at 0.0
audio_frame at 0.02
video_frame at 0.04
audio_frame at 0.05
audio_frame at 0.07
video_frame at 0.08
audio_frame at 0.09
audio_frame at 0.12
video_frame at 0.13
audio_frame at 0.14
audio_frame at 0.16
video_frame at 0.17
audio_frame at 0.19
video_frame at 0.21
audio_frame at 0.21
audio_frame at 0.23
video_frame at 0.25
audio_frame at 0.26
audio_frame at 0.28
video_frame at 0.29
audio_frame at 0.3
audio_frame at 0.33
video_frame at 0.33
audio_frame at 0.35
audio_frame at 0.37
video_frame at 0.38
audio_frame at 0.39
video_frame at 0.42
audio_frame at 0.42
audio_frame at 0.44
video_frame at 0.46
audio_frame at 0.46
audio_frame at 0.49
audio_frame at 0.51
audio_frame at 0.53
video_frame at 0.54
audio_frame at 0.56
audio_frame at 0.58
video_frame at 0.58
audio_frame at 0.6
video_frame at 0.63
audio_frame at 0.63
audio_frame at 0.65
video_frame at 0.67
audio_frame at 0.67
Time stamp should be updated according to the audio time base (implemented)
Some videos are already encoded with their correct pts, but the sort method I used will not affect those files.
But sometimes it may show random stuttering with videos having variable frame rates, because we can't add all the frames at once (memory issue), and we can't seek back the decoding process, so some frames must be dropped.
Other than this method, there is no good method to sync both audio/video with pyav. Moreover, one benefit of using pyaudio with the correct sample rate helps us avoid using the sleep method, you can treat the audio stream as the video clock.
Ok, thanks for making this request, we'll keep this in beta for now. I just want one more confirmation from a developer using this library stating this has no major problems. I'll merge as soon as I get confirmation from one more dev,
