How Voice Assistants Know When You've Finished Talking
This patent describes a method for speech recognition systems to accurately detect the end of a spoken phrase by considering multiple interpretations of the audio and how much silence each interpretation suggests.
Original patent title: “Language model speech endpointing”
This patent describes a method for speech recognition systems to accurately detect the end of a spoken phrase by considering multiple interpretations of the audio and how much silence each interpretation suggests. Granted to Amazon Technologies in 2018 with 25 claims and 10 forward citations, and it is expected to expire in 2035.
Coverage
What does this patent actually cover?
This patent details a computer-implemented method for determining when a speaker has finished talking during automatic speech recognition (ASR) processing. It works by receiving audio data, like from a mobile device microphone, and then performing ASR to generate a "plurality of hypotheses" (ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1). Each hypothesis is a different guess about what was said. For each guess, the system figures out its probability and how many "non-speech audio frames" (silence or background noise) immediately came before a potential end point. It then calculates a "respective score" for each hypothesis by multiplying its probability by a factor related to its non-speech frames. These scores are summed to create a "cumulative score." If this cumulative score goes above a certain "first threshold," the system decides that the speaker has likely finished speaking at that point. For example, if you say "Hey Alexa, what's the weather?" the system constantly checks if the weighted sum of non-speech from all its guesses indicates you've stopped talking after "weather."
The gap
What does this patent NOT cover?
- Does not cover endpointing methods that rely solely on a single best guess (hypothesis) of what was said, rather than considering multiple possibilities.
- Does not cover systems that only detect silence or a drop in audio volume without considering the linguistic context or probabilities of different interpretations.
- Does not cover endpointing based on a fixed duration of silence after detected speech, without weighting by hypothesis probability.
- Does not cover endpointing methods that do not use a "plurality of hypotheses" from an ASR decoder.
- Does not cover systems that determine an endpoint based on a user pressing a button to indicate the end of speech.
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
Key facts
What made this novel
The cleverness lies in not just looking for silence, but in considering multiple possible interpretations (hypotheses) of the audio simultaneously. By weighting the amount of non-speech each hypothesis suggests by its probability, the system gets a more robust and intelligent estimate of when speech truly ends, even amidst pauses or background noise.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
Amazon Alexa voice assistant
Google Assistant
Apple Siri
Microsoft Cortana
Voice control in smart home devices
Transcription services using ASR
Why it matters
The bigger picture
Accurate speech endpointing is crucial for voice-controlled devices. If a system cuts off too early, it misses part of the command. If it waits too long, the user experiences frustrating delays. This patent helps voice assistants like Amazon's Alexa or Google Assistant respond quickly and correctly, making interactions feel more natural and efficient. It addresses a fundamental challenge in making voice interfaces reliable for everyday use.
Filed
June 29, 2015
Granted
November 6, 2018
Market context
Who's building on this
Companies in this space
Amazon Technologies Inc., the assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →, continues to be a major player in this field, constantly refining its Alexa voice assistant and underlying ASR technology. Other large tech companies like Google, Apple, and Microsoft are also heavily invested in improving speech recognition and endpointing for their respective voice assistants and cloud AI services.
Market impact
This technology has significantly contributed to the improved responsiveness and accuracy of voice assistants, which are now ubiquitous in smart homes and mobile devices. By providing a more reliable way to detect the end of speech, it helped reduce user frustration and accelerate the adoption of voice-controlled interfaces, enabling entirely new product categories and interaction paradigms.
Claim 1 — Plain English
What this patent covers
This patent details a computer-implemented method for determining when a speaker has finished talking during automatic speech recognition (ASR) processing. It works by receiving audio data, like from a mobile device microphone, and then performing ASR to generate a "plurality of hypotheses" (Claim 1). Each hypothesis is a different guess about what was said. For each guess, the system figures out its probability and how many "non-speech audio frames" (silence or background noise) immediately came before a potential end point. It then calculates a "respective score" for each hypothesis by multiplying its probability by a factor related to its non-speech frames. These scores are summed to create a "cumulative score." If this cumulative score goes above a certain "first threshold," the system decides that the speaker has likely finished speaking at that point. For example, if you say "Hey Alexa, what's the weather?" the system constantly checks if the weighted sum of non-speech from all its guesses indicates you've stopped talking after "weather."
The clever bit
The cleverness lies in not just looking for silence, but in considering multiple possible interpretations (hypotheses) of the audio simultaneously. By weighting the amount of non-speech each hypothesis suggests by its probability, the system gets a more robust and intelligent estimate of when speech truly ends, even amidst pauses or background noise.
What it does not cover
- Does not cover endpointing methods that rely solely on a single best guess (hypothesis) of what was said, rather than considering multiple possibilities.
- Does not cover systems that only detect silence or a drop in audio volume without considering the linguistic context or probabilities of different interpretations.
- Does not cover endpointing based on a fixed duration of silence after detected speech, without weighting by hypothesis probability.
- Does not cover endpointing methods that do not use a "plurality of hypotheses" from an ASR decoder.
- Does not cover systems that determine an endpoint based on a user pressing a button to indicate the end of speech.
Patent timeline
Application submitted to the patent office
Application published, typically 18 months after filing
Patent officially issued
Patent enters public domain
PatentBrief Score
Impact Score
Strong
Citation count
21/40
Early citations
Claim breadth
17/20
Very broad protection
Recency
10/20
Granted 5–10 years ago
Assignee scale
20/20
Major company or institution
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$117K – $374K
Midpoint $234K · 8.7 yr remaining · industry ×1.5
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
Claim text not yet imported for this patent
The original legal language
Original claims
25 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
Rastrow, A., Hoffmeister, B., & Liu, B. (2018). How Voice Assistants Know When You've Finished Talking (U.S. Patent No. 10,121,471). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/10121471/language-model-speech-endpointing
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US10121471"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4965188 · 1990
How to Make Many Copies of a DNA Piece with Heat
This patent describes the Polymerase Chain Reaction (PCR) method, a technique to make millions of copies of a specific DNA segment using a heat-resistant enzyme and repeated temperature changes.
Cetus Corp
US 4235871 · 1980
How to Encapsulate Active Materials in Lipid Bubbles Efficiently
This patent describes a method for trapping biologically active substances inside tiny, multi-layered fat bubbles called liposomes, using a specific water-in-oil emulsion and gel-forming process to improve how much material gets captured.
Individual
Semantically similar
You might also find these interesting
US 10923111 · 2021 · Amazon Technologies
How Voice Assistants Distinguish Speakers After a Wakeword
US 10847149 · 2020 · Amazon Technologies
How Voice Assistants Listen for More Commands Without a Wake Word
US 11361763 · 2022 · Amazon Technologies
How Smart Speakers Know You're Talking to Them After a Command
US 11514901 · 2022 · Amazon Technologies
How Voice Assistants Recognize Who is Talking
More to explore
More in Software & Internet
US 4405829 · 1983 · Massachusetts Institute of Technology
How RSA Public-Key Encryption Keeps Digital Messages Secret
US 6285999 · 2001 · Leland Stanford Junior University
How Websites Get Ranked by Importance
US 5960411 · 1999 · Amazon com Inc
How Amazon's One-Click Ordering Works for Online Purchases
US 7669123 · 2010 · Facebook Inc
Displaying Friends' Activities in a Social Network Feed
New to patents?
Common Questions
Frequently Asked Questions
What does How Voice Assistants Know When You've Finished Talking cover?
This patent describes a method for speech recognition systems to accurately detect the end of a spoken phrase by considering multiple interpretations of the audio and how much silence each interpretation suggests.
Who owns patent US 10121471?
Amazon Technologies owns this patent, granted in 2018.
When does this patent expire?
This patent is expected to expire on June 29, 2035, when the invention enters the public domain.
What is patent US 10121471 cited by?
This patent has been cited by 10 later patents that build on its ideas.
What problem does this patent solve?
Accurate speech endpointing is crucial for voice-controlled devices. If a system cuts off too early, it misses part of the command. If it waits too long, the user experiences frustrating delays. This patent helps voice assistants like Amazon's Alexa or Google Assistant respond quickly and correctly, making interactions feel more natural and efficient. It addresses a fundamental challenge in making voice interfaces reliable for everyday use.
What does this patent NOT cover?
Does not cover endpointing methods that rely solely on a single best guess (hypothesis) of what was said, rather than considering multiple possibilities.
Same assignee
More from Amazon Technologies
Patent monitoring