Audio hiccups and cuts with Gemini Live API in LiveKit

[Omarimai]

Audio hiccups and cuts with Gemini Live API in LiveKit

Description

I'm experiencing audio playback issues when using the Gemini Live API with LiveKit. The audio has noticeable hiccups and cuts during playback.

Debugging findings

Through debugging, I've identified that the issue appears to be related to the timing of audio data delivery:

• The Gemini Live API sends audio in bursts
• The audio player attempts to consume the buffered audio
• When the player reaches the end of a burst before the next burst arrives, it causes audible hiccups/cuts in playback

Question

Is anyone else encountering similar audio discontinuity issues when using Gemini Live API with LiveKit? Would implementing a jitter buffer in the plugin help mitigate this issue?

Environment

• LiveKit version : 1.0.13
• google-genai : 1.39.0
• Transport: SIP, Web, etc.

Potential solutions

I'm wondering if a jitter buffer implementation within the plugin would be the appropriate solution to smooth out these bursty audio transmissions. Any guidance or recommendations would be appreciated.

[fahadanwaar]

Please team response, we are facing the same issues on our side as well.

[khushalcodiste]

Same issues with elevenlabs

[drishi1990]

same issue faced by us as well , is there any update

[davidzhao]

Gemini Live sometimes doesn't produce audio as fast as real-time. when that happens, the audio will glitch out momentarily due to having insufficient data.

There isn't really a way to fix this from the framework's perspective. We expect models to produce data faster than playback.