Streaming
Design low-latency transcription around interim events, finalized segments, backpressure, and reconnection, for live captions and voice agents.
Streaming ASR processes audio while it is being produced. It is the foundation for live captions, in-call intelligence, and future real-time voice-agent APIs.
Event lifecycle
A streaming integration should distinguish:
- Interim text, which can change as more audio arrives
- Finalized segments, which are stable enough to store or show as committed text
- Session metadata, including model and processing state
- Terminal state, indicating a clean finish or an error
Render interim text optimistically but persist finalized segments.
Production considerations
- Match the required audio encoding and framing described by the access-specific API reference.
- Bound client-side buffers and apply backpressure instead of accumulating unbounded audio.
- Expect network interruptions and design explicit reconnect behavior.
- Keep a local session identifier so application state can be reconciled after a disconnect.
- Measure time to first text and time to final text separately.
Downstream processing
Some processors can operate incrementally; others require a finalized segment or completed recording. Design the experience so slower understanding stages do not block immediate transcript delivery.
Future voice-agent APIs will build on this low-latency foundation with interruption handling and action execution.
Models
Choose between Ultrafield's proprietary models and open-weight options such as Whisper large-v3, weighing accuracy, latency, languages, and cost.
Speaker diarization
Separate a recording into speaker turns and align speaker labels with the transcript, either on its own or as one stage of a pipeline.