agents icon indicating copy to clipboard operation
agents copied to clipboard

feat(whisper): add Whisper plugin for LiveKit

Open imsakg opened this issue 11 months ago • 3 comments

Introduce the Whisper plugin for LiveKit, enabling offline speech-to-text capabilities using local Whisper model inference. This includes:

  • Initial setup of the plugin structure with classes for Whisper model, speech-to-text processing, and audio utilities.
  • Integration with essential libraries like numpy, ctranslate2, and faster_whisper for enhanced audio processing and transcription.
  • Setup files (setup.py, pyproject.toml) for building and packaging the plugin.

imsakg avatar Feb 14 '25 09:02 imsakg

⚠️ No Changeset found

Latest commit: a10867d5d68d3c571f09b0c3bf9c7b656ea4bf88

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

changeset-bot[bot] avatar Feb 14 '25 09:02 changeset-bot[bot]

Introduce the Whisper plugin for LiveKit, enabling offline speech-to-text capabilities using local Whisper model inference. This includes:

* Initial setup of the plugin structure with classes for Whisper model, speech-to-text processing, and audio utilities.

* Integration with essential libraries like `numpy`, `ctranslate2`, and `faster_whisper` for enhanced audio processing and transcription.

* Setup files (`setup.py`, `pyproject.toml`) for building and packaging the plugin.

This helps a lot, thanks!

skiiruu avatar Feb 26 '25 15:02 skiiruu

Hey, this is awesome, we're looking into merging this after we release v1.0! We're going to expose our inference process API so we make sure to load the model only once is multiprocessing scenario

theomonnom avatar Mar 10 '25 08:03 theomonnom