Severe voice quality drop if record/observability is enabled (via API or via dashboard)
Bug Description
Following https://docs.livekit.io/agents/observability/, if we set recording to true (either through the webpage, or through session.start() parameter), the agent voice will sound really bad (also stretched 2x longer).
When repro'd, I could see this error logs
Exception in thread recorder_io_encode_thread:
Traceback (most recent call last):
File "/Users/ted/.local/share/uv/python/cpython-3.12.11-macos-aarch64-none/lib/python3.12/threading.py", line 1075, in _bootstrap_inner
self.run()
File "/Users/ted/.local/share/uv/python/cpython-3.12.11-macos-aarch64-none/lib/python3.12/threading.py", line 1012, in run
self._target(*self._args, **self._kwargs)
File "/Users/ted/Developer/livekit-worker/.venv/lib/python3.12/site-packages/livekit/agents/voice/recorder_io/recorder_io.py", line 243, in _encode_thread
for packet in stream.encode(av_frame):
^^^^^^^^^^^^^^^^^^^^^^^
File "av/audio/stream.pyx", line 17, in av.audio.stream.AudioStream.encode
File "av/audio/stream.pyx", line 26, in av.audio.stream.AudioStream.encode
File "av/codec/context.pyx", line 398, in av.codec.context.CodecContext.encode
File "av/codec/context.pyx", line 383, in av.codec.context.CodecContext._prepare_and_time_rebase_frames_for_encode
File "av/codec/context.pyx", line 220, in av.codec.context.CodecContext.open
File "av/error.pyx", line 428, in av.error.err_check
av.error.ExperimentalError: [Errno 733130664] Experimental feature
Expected Behavior
Recording shouldn't impact voice quality
Reproduction Steps
This was reproducible with two ways:
- With 1.3.2, I could repro with turning on the option on the webpage. Didnt try the session.start() route.
- After 1.3.3, somehow it's the default behavior in dev mode until I turn record=false off session.start(). This is weird though since the PR making that default value change is already included in 1.3.1. Something was preventing this before 1.3.3 I guess.
Operating System
macOS
Models Used
No response
Package Versions
livekit-agent=1.3.2
Session/Room/Call IDs
No response
Proposed Solution
Additional Context
No response
Screenshots and Recordings
No response
Same problem here, when "record=True" is set, tts voices are being corrupted.
We have a PR #4098 that should fix this (similar issue #4048)
this should be fixed in #4098. feel free to reopen the issue if not, please also attach the audio recording if possible.