Realtime API¶
realtime ¶
Realtime voice: a provider-neutral seam plus an OpenAI adapter.
The browser holds the audio connection to the provider; the server mints
short-lived credentials, reads relayed events and writes client events
through a :class:RealtimeProvider. See docs/concepts/realtime.md.
OpenAIRealtime ¶
:class:~orqest.realtime.RealtimeProvider for OpenAI Realtime over WebRTC.
http is used as given and never closed here; without one, each call
opens and closes its own client. ttl_s is the client secret's lifetime
(the provider allows 10–7200 s); it only has to outlive connecting.
Source code in orqest/realtime/openai.py
session_payload ¶
The session object sent when minting a client secret.
Source code in orqest/realtime/openai.py
AssistantTranscript ¶
What the model said, from its own output transcript.
ClientSession ¶
A short-lived credential for the browser.
The browser posts its SDP offer to connect_url with
Authorization: Bearer <client_secret> before expires_at. The
long-lived key never leaves the server.
PlaybackCleared ¶
The response's audio still to be played was dropped (the user talked over it, or the client cut it off): the user didn't hear all of it.
PlaybackStopped ¶
The response's audio has all been played out: the user has heard all of it.
Comes after the response's ResponseDone, which marks the end of
generation, not of playback. Not every transport reports playback.
ProviderError ¶
The provider reported an error on the event channel.
RealtimeError ¶
A provider call made by the server failed or was refused.
RealtimeProvider ¶
One realtime voice provider, seen from the server.
The client-event builders return provider wire events for the browser to send on its event channel; the server never holds the audio.
create_client_session
async
¶
hangup
async
¶
parse_event ¶
Translate one provider event. Unknown or malformed events give
None; this never raises.
scripted_line ¶
tool_result ¶
The client events that hand a function result back and, with
respond, let the model continue. respond=False only records
the result: the model says nothing more about it until the next turn
(for a result it has already acknowledged).
Source code in orqest/realtime/types.py
context_update ¶
cost_usd ¶
RealtimeUsage ¶
Tokens used by one response.
audio_in and text_in exclude cached input tokens; cached_in
counts those, since they are priced separately.
ResponseDone ¶
A response ended.
status says how: completed, cancelled (for example the user
talked over it), failed or incomplete (cut short, or a status the
adapter didn't recognise). cancelled is kept as a shorthand for
status == "cancelled"; give either one, and both must agree.
line_id is set when the response was a scripted line, so its
transcript can be checked against the words asked for.
SessionConfig ¶
Everything a session starts with. language=None leaves
transcription free to recognise any language.
The last three fields are optional tuning, sent only when set (otherwise
the provider's defaults apply): how much the model reasons before
answering (less is faster), a cap on output tokens per response, and
microphone noise reduction for a close (near_field, a phone or
headset) or distant (far_field, a laptop or room) microphone.
SpeechStarted ¶
The provider heard the user start speaking.
item_id is the user item this speech becomes, the same id its
UserTranscript carries, so a transcript can be tied to when its speech
began (transcripts arrive late, often after later speech started).
None when the provider didn't say.
SpeechStopped ¶
The provider heard the user stop speaking; item_id as for
:class:SpeechStarted.
ToolCall ¶
The model asked for a function call; arguments is already parsed.
ToolDef ¶
A function the model may call. parameters is a JSON Schema object.
TurnDetection ¶
How the provider decides that the user started and stopped talking.
server is a voice-activity detector tuned by threshold (0–1, higher
needs louder audio), prefix_ms (audio kept from before speech began)
and silence_ms (quiet that ends a turn). semantic lets the model
judge the end of a turn, tuned by eagerness.
interrupt_on_speech_only asks the adapter to keep a short non-speech
sound, such as a cough, from cancelling a response in progress, using
whatever the provider offers for that.
UserTranscript ¶
What the user said, as transcribed. Deltas have final=False.