Multilingual Voice Pipeline

A rider's spoken query flows ASR to intent to tool dispatch to response to speech within a tight latency budget

Multilingual Voice Pipeline A rider's spoken query flows ASR to intent to tool dispatch to response to speech within a tight latency budget 01 / Capture 02 / Recognize 03 / Understand 04 / Act + Respond 05 / Speak Rider Audio · spoken query · 01 / Capture · voice in Rider Audio spoken query voice in ASR · speech to text · 02 / Recognize ASR speech to text Intent NLU · language + intent · 03 / Understand · multilingual Intent NLU language + intent multilingual Tool Dispatch · service call · 04 / Act + Respond Tool Dispatch service call Response Gen · compose reply · 04 / Act + Respond Response Gen compose reply TTS · text to speech · 05 / Speak · voice out TTS text to speech voice out audio stream transcript intent + slots service data reply text Legend primary data policy / PII async batch data store

Latency Budget

  • • Every stage runs against a tight speech-to-speech deadline
  • • Streaming keeps the rider from waiting on full transcripts

Multilingual

  • • ASR and NLU detect and handle the rider's language
  • • Response is generated and spoken back in that language

Data-flow diagram • Built with Archify • Create yours ↗ • Hover to trace • R route • Click to focus • +/− zoom • M radar • [/] views • P play story • T theme • E export