A Voice Conversation Turn

Streaming listen, streaming respond, and barge-in interruption

A Voice Conversation Turn Streaming listen, streaming respond, and barge-in interruption speaks stream audio final transcript (endpoint) stream tokens stream audio chunks plays response interrupts (barge-in) cancel synthesis cancel generation new utterance Listen Respond Barge-in User · speaking · Sequence participant User speaking Client · capture + playback · Sequence participant Client capture + playback ASR · streaming STT · Sequence participant ASR streaming STT LLM · the brain · Sequence participant LLM the brain TTS · streaming voice · Sequence participant TTS streaming voice Legend request return security async trace

Listen incrementally

  • • Audio streams to ASR as the user talks
  • • VAD endpointing decides the turn is over
  • • The final transcript triggers the response

Respond incrementally

  • • The LLM streams tokens, not a finished answer
  • • TTS synthesizes those tokens as they arrive
  • • The user hears speech within a fraction of a second

Barge-in

  • • The user can talk over the agent
  • • Interruption cancels both generation and synthesis
  • • The agent stops mid-sentence and starts listening again

Sequence diagram • Built with Archify • Create yours ↗ • Hover to trace • R route • Click to focus • +/− zoom • M radar • [/] views • P play story • T theme • E export