Audio and Speech
Sound is the modality that makes AI conversational — the difference between typing to a machine and talking to it. And the same architectural ideas that transformed text and vision transformed audio too: treat the waveform as a sequence, train at scale, and one model can transcribe speech across languages, or synthesize a natural-sounding voice from text. Understanding how AI handles audio — recognition, synthesis, and understanding — completes the picture of the core modalities and shows how general the multimodal recipe has become.
Sound is the modality that makes AI conversational — the difference between typing to a machine and talking to it. The same architectural ideas that transformed text and vision transformed audio too: treat the waveform as a sequence, train at scale, and one model can transcribe speech across languages, or synthesize a natural voice from text.