How to Automatically Detect and Fix Changes in AI Model Data
This patent describes a system that automatically notices when the real-world data an AI model sees changes, causing its predictions to become less accurate, and then fixes the model.
Original patent title: “Systems and methods for detecting data drift for data used in machine learning models”
This patent describes a system that automatically notices when the real-world data an AI model sees changes, causing its predictions to become less accurate, and then fixes the model. Granted to Capital One Services in 2020 with 22 claims and 32 forward citations, and it is expected to expire in 2038.
Key facts
Coverage
What does this patent actually cover?
The system detects when the data used by a machine learning model changes over time, a problem called 'data drift'. It does this by first receiving model training data and generating a predictive model. Then, it takes new model input data to generate predicted data. Crucially, it receives 'event data' (real-world outcomes) and compares a 'data profile' of the predicted data to a data profile of the event data (ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1). If these profiles differ significantly, indicating data drift, the system then automatically corrects the model. For example, a credit card fraud detection model might be trained on past transaction data. If new types of fraud emerge, the system would compare the model's fraud predictions (predicted data) with actual confirmed fraud cases (event data) to see if the patterns have shifted. If drift is detected, the system could retrain the model using the newer event data (Claim 13) or adjust its internal settings, called hyperparameters (Claim 9).
The gap
What does this patent NOT cover?
- Does not cover detecting data drift without comparing the *profile* of predicted data to the *profile* of real-world event data, as specified in ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1.
- Does not cover systems that only detect data drift but do not automatically initiate a correction of the model, as 'correcting the model' is a required step in ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1.
- Does not cover detecting data drift based solely on changes in the input data *before* it's processed by the model, without considering the model's predictions or real-world outcomes (event data).
- Does not cover detecting data drift in models that are not 'predictive models', as the claimsclaimsThe numbered statements at the end of a patent that legally define what the inventor owns.Read more → consistently refer to this specific type.
- Does not cover methods of drift detection that do not involve receiving 'event data' for comparison with predicted data.
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
What made this novel
The noveltynoveltyThe requirement that an invention be different from anything publicly known before its priority date.Read more → lies in the system's ability to automatically compare the *data profile* of a model's predictions with the *data profile* of actual real-world outcomes (event data) to detect drift, and then to automatically correct the model. This moves beyond simply monitoring model performance to understanding *why* performance might be dropping and taking proactive steps to fix it.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
Credit card fraud detection systems
Loan application risk assessment models
Algorithmic trading systems
Predictive maintenance for industrial equipment
Recommendation engines on streaming services
Medical diagnostic AI tools
Why it matters
The bigger picture
Machine learning models are increasingly used in critical applications, from finance to healthcare. Their performance can degrade significantly if the real-world data they encounter changes from what they were trained on. This patent addresses a fundamental challenge in maintaining reliable AI systems by automating the detection and correction of such changes. This capability is essential for ensuring that AI models remain accurate and trustworthy over time, especially in dynamic environments.
Filed
October 26, 2018
Granted
March 24, 2020
Market context
Who's building on this
Companies in this space
Major cloud providers like Amazon Web Services (AWS SageMaker), Google Cloud (AI Platform), and Microsoft Azure (Azure ML) offer services for model monitoring and drift detection. Specialized MLOps platforms such as Databricks, MLflow, Arize AI, and WhyLabs are also actively developing and deploying similar capabilities. Capital One, the assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →, continues to build advanced AI systems for financial services, making this technology central to their operations.
Market impact
This type of technology has become a cornerstone of modern MLOps (Machine Learning Operations) practices. It enables companies to deploy and maintain AI models in production with greater confidence, reducing the need for constant manual oversight and preventing costly errors that arise from degraded model performance. Its widespread adoption has made AI systems more robust and reliable, fostering trust in AI-driven decision-making across various industries.
Claim 1 — Plain English
What this patent covers
The system detects when the data used by a machine learning model changes over time, a problem called 'data drift'. It does this by first receiving model training data and generating a predictive model. Then, it takes new model input data to generate predicted data. Crucially, it receives 'event data' (real-world outcomes) and compares a 'data profile' of the predicted data to a data profile of the event data (Claim 1). If these profiles differ significantly, indicating data drift, the system then automatically corrects the model. For example, a credit card fraud detection model might be trained on past transaction data. If new types of fraud emerge, the system would compare the model's fraud predictions (predicted data) with actual confirmed fraud cases (event data) to see if the patterns have shifted. If drift is detected, the system could retrain the model using the newer event data (Claim 13) or adjust its internal settings, called hyperparameters (Claim 9).
The clever bit
The novelty lies in the system's ability to automatically compare the *data profile* of a model's predictions with the *data profile* of actual real-world outcomes (event data) to detect drift, and then to automatically correct the model. This moves beyond simply monitoring model performance to understanding *why* performance might be dropping and taking proactive steps to fix it.
What it does not cover
- Does not cover detecting data drift without comparing the *profile* of predicted data to the *profile* of real-world event data, as specified in Claim 1.
- Does not cover systems that only detect data drift but do not automatically initiate a correction of the model, as 'correcting the model' is a required step in Claim 1.
- Does not cover detecting data drift based solely on changes in the input data *before* it's processed by the model, without considering the model's predictions or real-world outcomes (event data).
- Does not cover detecting data drift in models that are not 'predictive models', as the claims consistently refer to this specific type.
- Does not cover methods of drift detection that do not involve receiving 'event data' for comparison with predicted data.
Patent timeline
Application submitted to the patent office
Application published, typically 18 months after filing
Patent officially issued
Patent enters public domain
PatentBrief Score
Impact Score
Moderate
Citation count
30/40
Moderately cited
Claim breadth
15/20
Broad claimsclaimsThe numbered statements at the end of a patent that legally define what the inventor owns.Read more →
Recency
10/20
Granted 5–10 years ago
Assignee scale
0/20
Independent or smaller assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$281K – $899K
Midpoint $562K · 12.4 yr remaining · industry ×1.6
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
The original legal language
Original claims
22 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
Walters, A., Key, K., Watson, M., Goodsitt, J., Pham, V., Truong, A., Farivar, R., & Abad, F. A. T. (2020). How to Automatically Detect and Fix Changes in AI Model Data (U.S. Patent No. 10,599,957). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/10599957/systems-and-methods-for-detecting-data-drift-for-data-used-in-machine-learning-m
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US10599957"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4405829 · 1983
How RSA Public-Key Encryption Keeps Digital Messages Secret
This patent describes the foundational RSA algorithm, a method for securely sending messages where anyone can encrypt a message using a public key, but only the intended recipient can decrypt it using a secret private key.
Massachusetts Institute of Technology
US 4575330 · 1986
How 3D Printers Build Objects Layer by Layer from Liquid
This patent describes the foundational method for 3D printing, where a machine builds a three-dimensional object layer by layer by hardening a liquid material with light or other energy.
UVP Inc
Semantically similar
You might also find these interesting
US 12518214 · 2026 · Nant Holdings IP
Training AI on Private Data Without Seeing It
US 10824959 · 2020 · Amazon Technologies Inc
How to Make Artificial Intelligence Explain Its Own Decisions
US 12438891 · 2025 · Cisco Technology
How Multiple AI Models Detect Unusual Behavior on Computer Networks
US 11606373 · 2023 · Darktrace Holdings
AI System That Learns Normal Email Use to Spot and Stop Cyber Threats
More to explore
More in Software & Internet
US 4405829 · 1983 · Massachusetts Institute of Technology
How RSA Public-Key Encryption Keeps Digital Messages Secret
US 6285999 · 2001 · Leland Stanford Junior University
How Websites Get Ranked by Importance
US 5960411 · 1999 · Amazon com Inc
How Amazon's One-Click Ordering Works for Online Purchases
US 7669123 · 2010 · Facebook Inc
Displaying Friends' Activities in a Social Network Feed
New to patents?
Common Questions
Frequently Asked Questions
What does How to Automatically Detect and Fix Changes in AI Model Data cover?
This patent describes a system that automatically notices when the real-world data an AI model sees changes, causing its predictions to become less accurate, and then fixes the model.
Who owns patent US 10599957?
Capital One Services owns this patent, granted in 2020.
When does this patent expire?
This patent is expected to expire on October 26, 2038, when the invention enters the public domain.
What is patent US 10599957 cited by?
This patent has been cited by 32 later patents that build on its ideas.
What problem does this patent solve?
Machine learning models are increasingly used in critical applications, from finance to healthcare. Their performance can degrade significantly if the real-world data they encounter changes from what they were trained on. This patent addresses a fundamental challenge in maintaining reliable AI systems by automating the detection and correction of such changes. This capability is essential for ensuring that AI models remain accurate and trustworthy over time, especially in dynamic environments.
What does this patent NOT cover?
Does not cover detecting data drift without comparing the *profile* of predicted data to the *profile* of real-world event data, as specified in Claim 1.
Patent monitoring