openai-node icon indicating copy to clipboard operation
openai-node copied to clipboard

Type error on `session.audio.input.transcription.model` with custom development name on Azure's Realtime API

Open heguro opened this issue 5 months ago • 0 comments

Confirm this is a Node library issue and not an underlying OpenAI API issue

  • [x] This is an issue with the Node library

Describe the bug

In Azure's Realtime API, you need to specify a user-defined deployment name for session.audio.input.transcription.model, not a model name. However, in the GA API, the model property's type is fixed to 'whisper-1' | 'gpt-4o-mini-transcribe' | 'gpt-4o-transcribe' | 'gpt-4o-transcribe-diarize', which causes a type error when specifying an arbitrary deployment name. https://github.com/openai/openai-node/blob/d1c87a385727678ac23095e3ca10ec527135cfcb/src/resources/realtime/realtime.ts#L38

In the previous Preview API types, the type for SessionUpdateEvent.Session.InputAudioTranscription["model"] was string, which allowed for custom names. https://github.com/openai/openai-node/blob/d1c87a385727678ac23095e3ca10ec527135cfcb/src/resources/beta/realtime/realtime.ts#L2375-L2379

It might be better to either change the type to string for the GA API as well or add (string & {}) to the union like here: https://github.com/openai/openai-node/blob/d1c87a385727678ac23095e3ca10ec527135cfcb/src/resources/realtime/realtime.ts#L3002-L3005

To Reproduce

TS Playground

Code snippets

import { OpenAI } from "openai";
import { OpenAIRealtimeWS } from "openai/realtime/ws";

const apiKey = "my-api-key";
const openAI = new OpenAI({
  apiKey,
  baseURL: "wss://[my-endpoint].openai.azure.com/openai/v1/realtime?model=[my-gpt-realtime-develoyment-name]",
});
const realtimeClient = new OpenAIRealtimeWS(
  {
    model: "my-gpt-realtime-develoyment-name",
    options: {
      headers: { "api-key": apiKey },
    },
  },
  openAI,
);
realtimeClient.send({
  type: "session.update",
  session: {
    type: "realtime",
    output_modalities: ["audio"],
    audio: {
      input: {
        transcription: {
          // Type '"my-gpt-4o-transcribe-deployment-name"' is not assignable to type '"gpt-4o-mini-transcribe" | "gpt-4o-transcribe" | "gpt-4o-transcribe-latest" | "whisper-1" | undefined'.
          model: "my-gpt-4o-transcribe-deployment-name",
        },
      },
      output: {
        voice: "cedar",
      },
    },
  },
});

OS

Debian 12 (bookworm)

Node version

Node v22.20.0

Library version

openai v6.8.0

heguro avatar Nov 04 '25 09:11 heguro