How to Make Artificial Intelligence Explain Its Own Decisions
A system that helps complex machine learning models explain why they made a specific decision by turning their data into simple, readable rules.
Original patent title: “Explainers for machine learning classifiers”
A system that helps complex machine learning models explain why they made a specific decision by turning their data into simple, readable rules. Granted to Amazon Technologies Inc in 2020 with 23 claims and 37 forward citations, and it is expected to expire in 2036.
Coverage
What does this patent actually cover?
This system solves the 'black box' problem in artificial intelligence, where a model makes a decision but cannot explain why. It takes the original data used to train the model and creates a 'transformed data set' that links specific input features to the model's final predictions. It then uses rule-mining algorithms to find patterns—essentially 'if-then' statements—that describe how the model behaves. When the model makes a new prediction, the system looks at these pre-calculated rules to provide a human-readable reason for that specific outcome.
The gap
What does this patent NOT cover?
- Does not cover models that do not use a training set of observation records.
- Does not cover explanations generated without using a rule-mining algorithm.
- Does not cover systems that explain decisions using non-rule-based methods like feature importance heatmaps or saliency maps.
- Does not cover real-time model retraining during the explanation generation process.
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
Key facts
What made this novel
Instead of trying to interpret the complex internal math of a neural network directly, it treats the model as an object to be studied, mining rules from its outputs just like you would mine data from a database.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
Amazon SageMaker Model Monitor
Automated credit scoring systems
AI-driven fraud detection services
Why it matters
The bigger picture
As AI is used for high-stakes decisions like loan approvals or medical diagnoses, regulators and users demand transparency. This patent provides a structured way for cloud-based AI services to offer 'explainability' as a feature, which is essential for building trust in automated systems. It helps companies comply with requirements like the 'right to an explanation' found in privacy laws.
Filed
February 16, 2016
Granted
November 3, 2020
Market context
Who's building on this
Companies in this space
Amazon Web Services (AWS) is the primary assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more → and continues to integrate these explainability features into their SageMaker platform. Other major cloud providers like Google Cloud and Microsoft Azure are also heavily invested in similar 'Explainable AI' (XAI) frameworks to compete for enterprise clients.
Market impact
This patent contributed to the standardization of 'Explainable AI' as a required component of enterprise machine learning platforms. It helped shift the industry from viewing AI as an opaque, proprietary secret to treating transparency as a necessary service feature for business adoption.
Claim 1 — Plain English
What this patent covers
This system solves the 'black box' problem in artificial intelligence, where a model makes a decision but cannot explain why. It takes the original data used to train the model and creates a 'transformed data set' that links specific input features to the model's final predictions. It then uses rule-mining algorithms to find patterns—essentially 'if-then' statements—that describe how the model behaves. When the model makes a new prediction, the system looks at these pre-calculated rules to provide a human-readable reason for that specific outcome.
The clever bit
Instead of trying to interpret the complex internal math of a neural network directly, it treats the model as an object to be studied, mining rules from its outputs just like you would mine data from a database.
What it does not cover
- Does not cover models that do not use a training set of observation records.
- Does not cover explanations generated without using a rule-mining algorithm.
- Does not cover systems that explain decisions using non-rule-based methods like feature importance heatmaps or saliency maps.
- Does not cover real-time model retraining during the explanation generation process.
Patent timeline
Application submitted to the patent office
Application published, typically 18 months after filing
Patent officially issued
Patent enters public domain
PatentBrief Score
Impact Score
Strong
Citation count
32/40
Moderately cited
Claim breadth
15/20
Broad claimsclaimsThe numbered statements at the end of a patent that legally define what the inventor owns.Read more →
Recency
10/20
Granted 5–10 years ago
Assignee scale
20/20
Major company or institution
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$187K – $599K
Midpoint $374K · 9.6 yr remaining · industry ×1.6
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
Claim text not yet imported for this patent
The original legal language
Original claims
23 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
Rao, S. S. H., & Chatterjee, B. K. (2020). How to Make Artificial Intelligence Explain Its Own Decisions (U.S. Patent No. 10,824,959). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/10824959/explainers-for-machine-learning-classifiers
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US10824959"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4965188 · 1990
How to Make Many Copies of a DNA Piece with Heat
This patent describes the Polymerase Chain Reaction (PCR) method, a technique to make millions of copies of a specific DNA segment using a heat-resistant enzyme and repeated temperature changes.
Cetus Corp
US 4235871 · 1980
How to Encapsulate Active Materials in Lipid Bubbles Efficiently
This patent describes a method for trapping biologically active substances inside tiny, multi-layered fat bubbles called liposomes, using a specific water-in-oil emulsion and gel-forming process to improve how much material gets captured.
Individual
Semantically similar
You might also find these interesting
US 11651227 · 2023 · SRI International Inc
How to Force AI to Follow Logical Rules During Training
US 10599957 · 2020 · Capital One Services
How to Automatically Detect and Fix Changes in AI Model Data
US 11615208 · 2023 · Capital One Services LLC
How Cloud Systems Automatically Create and Train AI Data Models
US 10289962 · 2019 · Google LLC
How to Shrink Large AI Models Using Knowledge Distillation
More to explore
More in AI & Machine Learning
US 10452978 · 2019 · Google LLC
How AI Models Understand Language Using 'Attention'
US 6523026 · 2003 · Huntsman International LLC
How Computers Find Hidden Connections Between Different Fields of Knowledge
US 11615208 · 2023 · Capital One Services LLC
How Cloud Systems Automatically Create and Train AI Data Models
US 10402750 · 2019 · Facebook Inc
How Facebook Uses Deep Learning to Predict What You Might Like
New to patents?
Common Questions
Frequently Asked Questions
What does How to Make Artificial Intelligence Explain Its Own Decisions cover?
A system that helps complex machine learning models explain why they made a specific decision by turning their data into simple, readable rules.
Who owns patent US 10824959?
Amazon Technologies Inc owns this patent, granted in 2020.
When does this patent expire?
This patent is expected to expire on February 16, 2036, when the invention enters the public domain.
What is patent US 10824959 cited by?
This patent has been cited by 37 later patents that build on its ideas.
What problem does this patent solve?
As AI is used for high-stakes decisions like loan approvals or medical diagnoses, regulators and users demand transparency. This patent provides a structured way for cloud-based AI services to offer 'explainability' as a feature, which is essential for building trust in automated systems. It helps companies comply with requirements like the 'right to an explanation' found in privacy laws.
What does this patent NOT cover?
Does not cover models that do not use a training set of observation records.
Same assignee
More from Amazon Technologies Inc
Patent monitoring