OpenAI Realtime API

The OpenAI Realtime API delivers low-latency, multimodal communication: enabling fluid, speech-to-speech interactions and real-time transcription via WebSockets and WebRTC.

This is the engine for truly conversational AI: a unified API designed for low-latency, speech-to-speech interactions. It bypasses multi-step processing, utilizing protocols like WebSockets and WebRTC to maintain an open connection for instantaneous exchanges. Running on natively multimodal models (e.g., GPT-4o), the API handles audio, images, and text seamlessly. Developers use it to build sophisticated voice agents, implement real-time transcription, and create dynamic, interruptible conversational experiences. Focus on the core: it provides the speed and flexibility necessary for modern, voice-first applications.

https://platform.openai.com/docs/guides/realtime/overview