openai
Using realtime API with python client
Confirm this is a feature request for the Python library and not the underlying OpenAI API.
- [X] This is a feature request for the Python library
Describe the feature or improvement you're requesting
I had fastAPI server running on my application already. I wanted to make it's user interface voice to voice type. On openai's official documentation only steps for node.js is given. I was wondering if I'd be able to reuse my existing code ( fastapi + openai python ) for voice interface too.
Additional context
Realtime API Docs : https://platform.openai.com/docs/guides/realtime
currently not sure how can I use openai-python for realtime API.
Microsoft has a python version you can use. Please note that there are connection differences between OpenAI and Azure OpenAI that you'll need to change, but the rest should work for you.
https://github.com/Azure-Samples/aoai-realtime-audio-sdk/tree/main/python
If I'm not wrong azure-openai does support openai directly. (From their implmentation it seems like they do)
they've method for getting messages with openai.
https://github.com/Azure-Samples/aoai-realtime-audio-sdk/blob/8105a5c3ab9cc54fe864aa6f8259f72c6829eec7/python/samples/low_level_sample.py#L251
Hi @hayescode I did try it was establishing the connection but it requires audio files path as input. If you have any information do you know how can I stream the audio directly to the client? Here what I have to do is save to audio file and then send to openai
You'll need a frontend to stream and decode the audio chunks. I use Chainlit which implements this differently but it's still python backend, if just you're looking for a quick POC try this:
https://github.com/Chainlit/cookbook/tree/main/realtime-assistant
This SDK now has builtin support for the Realtime API! https://github.com/openai/openai-python?tab=readme-ov-file#realtime-api-beta
