# How to Automatically Detect and Fix Changes in AI Model Data > This patent describes a system that automatically notices when the real-world data an AI model sees changes, causing its predictions to become less accurate, and then fixes the model. - **Patent:** US 10599957 - **Original title:** Systems and methods for detecting data drift for data used in machine learning models - **Owner:** Capital One Services - **Granted:** 2020 - **Status:** Active - **Times cited:** 32 - **Field:** software, ai_ml, finance, telecommunications, consumer_electronics ## What it does The system detects when the data used by a machine learning model changes over time, a problem called 'data drift'. It does this by first receiving model training data and generating a predictive model. Then, it takes new model input data to generate predicted data. Crucially, it receives 'event data' (real-world outcomes) and compares a 'data profile' of the predicted data to a data profile of the event data (Claim 1). If these profiles differ significantly, indicating data drift, the system then automatically corrects the model. For example, a credit card fraud detection model might be trained on past transaction data. If new types of fraud emerge, the system would compare the model's fraud predictions (predicted data) with actual confirmed fraud cases (event data) to see if the patterns have shifted. If drift is detected, the system could retrain the model using the newer event data (Claim 13) or adjust its internal settings, called hyperparameters (Claim 9). ## What it does NOT cover - Does not cover detecting data drift without comparing the *profile* of predicted data to the *profile* of real-world event data, as specified in Claim 1. - Does not cover systems that only detect data drift but do not automatically initiate a correction of the model, as 'correcting the model' is a required step in Claim 1. - Does not cover detecting data drift based solely on changes in the input data *before* it's processed by the model, without considering the model's predictions or real-world outcomes (event data). - Does not cover detecting data drift in models that are not 'predictive models', as the claims consistently refer to this specific type. - Does not cover methods of drift detection that do not involve receiving 'event data' for comparison with predicted data. ## The clever bit The novelty lies in the system's ability to automatically compare the *data profile* of a model's predictions with the *data profile* of actual real-world outcomes (event data) to detect drift, and then to automatically correct the model. This moves beyond simply monitoring model performance to understanding *why* performance might be dropping and taking proactive steps to fix it. ## Real-world examples 1. Credit card fraud detection systems 2. Loan application risk assessment models 3. Algorithmic trading systems 4. Predictive maintenance for industrial equipment 5. Recommendation engines on streaming services 6. Medical diagnostic AI tools ## Why it matters Machine learning models are increasingly used in critical applications, from finance to healthcare. Their performance can degrade significantly if the real-world data they encounter changes from what they were trained on. This patent addresses a fundamental challenge in maintaining reliable AI systems by automating the detection and correction of such changes. This capability is essential for ensuring that AI models remain accurate and trustworthy over time, especially in dynamic environments. ## Frequently asked questions ### What does How to Automatically Detect and Fix Changes in AI Model Data cover? This patent describes a system that automatically notices when the real-world data an AI model sees changes, causing its predictions to become less accurate, and then fixes the model. ### Who owns patent US 10599957? Capital One Services owns this patent, granted in 2020. ### When does this patent expire? This patent is expected to expire on October 26, 2038, when the invention enters the public domain. ### What is patent US 10599957 cited by? This patent has been cited by 32 later patents that build on its ideas. ### What problem does this patent solve? Machine learning models are increasingly used in critical applications, from finance to healthcare. Their performance can degrade significantly if the real-world data they encounter changes from what they were trained on. This patent addresses a fundamental challenge in maintaining reliable AI systems by automating the detection and correction of such changes. This capability is essential for ensuring that AI models remain accurate and trustworthy over time, especially in dynamic environments. ### What does this patent NOT cover? Does not cover detecting data drift without comparing the *profile* of predicted data to the *profile* of real-world event data, as specified in Claim 1. **Full plain-English explainer:** https://patentbrief.org/patent/us/10599957/systems-and-methods-for-detecting-data-drift-for-data-used-in-machine-learning-m **Original patent:** https://patents.google.com/patent/US10599957 --- _Source: PatentBrief — https://patentbrief.org. Patent facts are from public records; the plain-English explanation is PatentBrief's._ ## Related patents Semantically similar inventions in the PatentBrief corpus: - [Training AI on Private Data Without Seeing It](https://patentbrief.org/patent/us/12518214/distributed-machine-learning-systems-including-generation-of-synthetic-data) — This patent describes a way to train artificial intelligence models using private data stored on many separate computers, by generating fake data that mimics the real data's patterns, so the private data itself never leaves its original location. - [How to Make Artificial Intelligence Explain Its Own Decisions](https://patentbrief.org/patent/us/10824959/explainers-for-machine-learning-classifiers) — A system that helps complex machine learning models explain why they made a specific decision by turning their data into simple, readable rules. - [How Multiple AI Models Detect Unusual Behavior on Computer Networks](https://patentbrief.org/patent/us/12438891/anomaly-detection-based-on-ensemble-machine-learning-model) — This patent describes a computer system that uses several artificial intelligence models working together to spot unusual and potentially dangerous activity from users or devices on a computer network. - [AI System That Learns Normal Email Use to Spot and Stop Cyber Threats](https://patentbrief.org/patent/us/11606373/cyber-threat-defense-system-protecting-email-networks-with-machine-learning-mode) — This 2023 patent describes an AI system that learns how your company normally uses email and then automatically takes action to stop cyber threats that behave unusually. - [How to Save and Reuse Skills Learned by Artificial Intelligence Hardware](https://patentbrief.org/patent/us/10410117/method-and-a-system-for-creating-dynamic-neural-function-libraries) — A method for capturing the internal settings of a neuromorphic AI chip after it learns a task, allowing that 'skill' to be exported and loaded onto another AI chip.