PyAV-Org
Multiple streams leads to fail
Overview
Using multiple stream readers leads to runtime fail. All readers in my tests use the same source URL, I did not tried different because don't have tens of sources for a test.
Expected behavior
Work without runtime errors
Actual behavior
Random runtime errors
Traceback
I tried many different sources and with any source it fails randomly. Sometimes it works for a long time so you need to run test below again and again, sometimes fails fast.
But in any case fail starts with messages:
Stream ends prematurely at 896293, should be 1063655
...
[NULL @ 0xcc80040] Invalid NAL unit size (-131548183 > 20682).
[hevc @ 0xcedb2c0] Invalid NAL unit size (-131548183 > 20682).
[hevc @ 0xcedb2c0] Error splitting the input into NAL units.
...
[NULL @ 0xf87b000] Invalid NAL unit size (13281428 > 20682).
[hevc @ 0xfabb5c0] Invalid NAL unit size (13281428 > 20682).
[hevc @ 0xfabb5c0] Error splitting the input into NAL units
...
and ends usually with:
Traceback (most recent call last):
File "av/container/input.pyx", line 186, in decode
File "av/container/input.pyx", line 146, in demux
File "av/container/core.pyx", line 292, in av.container.core.Container.err_check
File "av/error.pyx", line 336, in av.error.err_check
av.error.ExitError: [Errno 1414092869] Immediate exit requested: [URL]
No other logs are available in these cases.
Investigation
Eventually I registered this problem with two readers in my code. After I understand problem in multiple instances, I've created minimal sample below.
First, I thought this happens only within one process or thread. But I tired to use multiprocessing (all the same, just readers implemented as process daemons with queue) and all the same - it fails with the same output.
Reproduction
Minimal sample to reproduce:
import av
av.logging.set_level(av.logging.DEBUG)
N_containers = 40
readers = []
for i in range(N_containers):
container1 = av.open(
'rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mp4',
options={
'live_start_index': '0',
'http_persistent': '1',
'http_seekable': '1',
},
)
streams_by_height = {s.height: s for s in container1.streams.video}
stream = streams_by_height[max(streams_by_height)]
stream.thread_count = 0
stream.thread_type = 1
readers.append(iter(container1.decode(stream)))
while True:
for reader in readers:
frame = next(reader)
It's quite weird but useful to show the problem.
Versions
- OS: Ubuntu 20.04.4 LTS
- PyAV runtime:
PyAV v9.2.0
library configuration: --prefix=/usr --extra-version='0ubuntu1~20.04.sav1.3' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libdav1d --enable-librist --enable-libvmaf --enable-libzimg --enable-crystalhd --enable-libmfx --enable-libsvtav1 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
library license: GPL version 2 or later
libavcodec 58.134.100
libavdevice 58. 13.100
libavfilter 7.110.100
libavformat 58. 76.100
libavutil 56. 70.100
libswresample 3. 9.100
libswscale 5. 9.100
- FFmpeg:
ffmpeg version 4.4.git Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
configuration: --enable-cuda --enable-nvenc --enable-cuvid
libavutil 57. 27.100 / 57. 27.100
libavcodec 59. 33.100 / 59. 33.100
libavformat 59. 25.100 / 59. 25.100
libavdevice 59. 6.100 / 59. 6.100
libavfilter 8. 41.100 / 8. 41.100
libswscale 6. 6.100 / 6. 6.100
libswresample 4. 6.100 / 4. 6.100
But, I've registered this problem with very different OS, ffmpeg and PyAv versions.
Research
I have done the following:
- [✔] Checked the PyAV documentation
- [✔] Searched on Google
- [✔] Searched on Stack Overflow
- [✔] Looked through old GitHub issues
- [No] Asked on PyAV Gitter
- [No] ... and waited 72 hours for a response.
