How Smart Speakers Know You're Talking to Them After a Command

This patent describes how a smart speaker system can tell if follow-up speech is meant for it, even without a "wake word," by analyzing voice activity and partial speech recognition results using an AI model.

What it covers

The patent details a method for a system, like a smart speaker, to identify if incoming speech is directed at it, especially after an initial interaction. First, the system receives an first command, processes it, and responds. Crucially, it then instructs the device to send subsequent audio without requiring a wake word (Claim 1). When this second audio arrives, the system first checks for voice activity. Then, it performs automatic speech recognition (ASR) on the audio. While ASR is running, it simultaneously creates a "feature vector" from the early parts of the ASR results and feeds this into a deep neural network (DNN). This DNN calculates a score indicating how likely the speech is intended for the system. If the score passes a certain level, the system proceeds to understand the full speech and act on it. For example, after you say "Alexa, set a timer for 10 minutes," the system might then listen for a follow-up like "and add a reminder for my meeting" without needing you to say "Alexa" again.

What it doesn't cover

• —Does not cover systems that always require a wake word for every interaction.
• —Does not cover systems that rely solely on voice activity detection to determine system-directed speech.
• —Does not cover determining system-directed speech without using a deep neural network on a feature vector derived from partial ASR results.
• —Does not cover systems where the device itself determines the presence of a wake word in the second input audio data (Claim 1 explicitly states "without the device determining a presence of a wakeword").
• —Does not cover systems that only use full ASR results, rather than partial ASR results, to create the feature vector for the DNN.

The clever bit

The innovation lies in using partial ASR results, combined with other audio characteristics, to predict if speech is system-directed in parallel with the main ASR process. This allows the system to quickly decide whether to fully process the speech or discard it, saving computational resources and improving responsiveness for follow-up commands.

Why it matters

This technology is fundamental for creating more natural and conversational interactions with voice assistants. It allows users to have follow-up conversations without repeatedly saying the wake word, making the experience smoother and less clunky. This capability is key to the user experience of modern smart speakers and virtual assistants, enabling multi-turn dialogues.

Real-world examples

• 1.Amazon Alexa's "Follow-Up Mode"
• 2.Google Assistant's "Continued Conversation"
• 3.Most modern smart speaker follow-up interactions