Improving AI Predictions by Factoring in How Much You Trust Each Model
This patent describes a method for making an AI system more accurate by combining several individual AI models and giving more weight to the ones considered more trustworthy.
Original patent title: “Ensemble machine learning models incorporating a model trust factor”
This patent describes a method for making an AI system more accurate by combining several individual AI models and giving more weight to the ones considered more trustworthy. Owned by Noblis with 34 claims and 31 forward citations, and it is expected to expire in 2041.
Coverage
What does this patent actually cover?
This patent describes a method for training an ensemble machine learning model, which is like a team of AI models working together. First, it receives data about how much to trust each individual AI model in the team (ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1a). This 'trust score' can be based on things like how sensitive the model is to bad input data or how confident its past predictions were (Claim 5). Next, it calculates an estimated prediction error for each model, but this error is adjusted by its trust score (Claim 1b). A less trusted model might have its errors penalized more. Then, it uses these adjusted error estimates to figure out a 'normalized weight' for each model (Claim 1c). Finally, it uses these normalized weights to determine the overall prediction equation for the entire ensemble, giving more influence to the more trusted models (Claim 1d). For example, if you have three AI models predicting stock prices, and one is known to be very reliable with certain types of data, this method would give that reliable model's predictions more say in the final combined forecast.
The gap
What does this patent NOT cover?
- Does not cover single machine learning models that do not combine predictions from multiple individual models.
- Does not cover ensemble models that combine predictions without explicitly calculating and using a 'trust score' for each individual model.
- Does not cover methods where the trust score does not influence the prediction error estimate or the normalized weights of the individual models.
- Does not cover trust scores that are not real numbers, specifically those outside the 0.0 to 1.0 range as described in ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 3.
- Does not cover ensemble methods where the output prediction equation is not determined, at least in part, by these normalized weights.
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
Key facts
What made this novel
The clever part is explicitly integrating a 'trust factor' into the mathematical calculation of how much each individual model's prediction contributes to the final combined output. Instead of just relying on past performance metrics, this method allows for a more nuanced understanding of a model's reliability to directly influence its weighting.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
Fraud detection systems combining multiple AI models
Medical diagnostic tools using various AI algorithms
Autonomous vehicle perception systems blending sensor data interpretations
Financial forecasting platforms integrating diverse predictive models
Cybersecurity threat detection systems
Why it matters
The bigger picture
As AI models become more complex and are used in critical applications like healthcare or finance, ensuring their reliability and accuracy is crucial. This patent offers a way to build more robust AI systems by explicitly incorporating a measure of trust into how multiple models collaborate. This can lead to more dependable predictions, especially when individual models might have varying levels of quality or applicability to different data.
Filed
December 21, 2021
Market context
Who's building on this
Companies in this space
Noblis Inc., the assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →, is a non-profit science, technology, and strategy organization that often works with government agencies. Companies focused on explainable AI (XAI) and trustworthy AI, such as IBM, Google, and Microsoft, are actively researching and developing methods to improve AI reliability and transparency. Startups specializing in AI governance and model risk management also operate in this space.
Market impact
This patent contributes to the growing field of trustworthy AI, which is critical for broader adoption of AI in regulated and high-stakes industries. By offering a structured way to incorporate model trust, it could lead to AI systems that are more resilient to individual model failures or biases. This could reduce risks associated with AI deployment and increase confidence among users and regulators, potentially influencing standards for AI system development and evaluation.
Claim 1 — Plain English
What this patent covers
This patent describes a method for training an ensemble machine learning model, which is like a team of AI models working together. First, it receives data about how much to trust each individual AI model in the team (Claim 1a). This 'trust score' can be based on things like how sensitive the model is to bad input data or how confident its past predictions were (Claim 5). Next, it calculates an estimated prediction error for each model, but this error is adjusted by its trust score (Claim 1b). A less trusted model might have its errors penalized more. Then, it uses these adjusted error estimates to figure out a 'normalized weight' for each model (Claim 1c). Finally, it uses these normalized weights to determine the overall prediction equation for the entire ensemble, giving more influence to the more trusted models (Claim 1d). For example, if you have three AI models predicting stock prices, and one is known to be very reliable with certain types of data, this method would give that reliable model's predictions more say in the final combined forecast.
The clever bit
The clever part is explicitly integrating a 'trust factor' into the mathematical calculation of how much each individual model's prediction contributes to the final combined output. Instead of just relying on past performance metrics, this method allows for a more nuanced understanding of a model's reliability to directly influence its weighting.
What it does not cover
- Does not cover single machine learning models that do not combine predictions from multiple individual models.
- Does not cover ensemble models that combine predictions without explicitly calculating and using a 'trust score' for each individual model.
- Does not cover methods where the trust score does not influence the prediction error estimate or the normalized weights of the individual models.
- Does not cover trust scores that are not real numbers, specifically those outside the 0.0 to 1.0 range as described in Claim 3.
- Does not cover ensemble methods where the output prediction equation is not determined, at least in part, by these normalized weights.
Patent timeline
Application submitted to the patent office
Patent enters public domain
PatentBrief Score
Impact Score
Moderate
Citation count
30/40
Moderately cited
Claim breadth
20/20
Very broad protection
Recency
0/20
Older than 20 years
Assignee scale
0/20
Independent or smaller assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$276K – $885K
Midpoint $553K · 15.3 yr remaining · industry ×1.6
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
Claim text not yet imported for this patent
The original legal language
Original claims
34 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
KALKA, N., Anderson, J., & BARTLOW, N. D. Improving AI Predictions by Factoring in How Much You Trust Each Model (U.S. Patent No. 20,230,044,102). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/20230044102/ensemble-machine-learning-models-incorporating-a-model-trust-factor
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US20230044102"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4965188 · 1990
How to Make Many Copies of a DNA Piece with Heat
This patent describes the Polymerase Chain Reaction (PCR) method, a technique to make millions of copies of a specific DNA segment using a heat-resistant enzyme and repeated temperature changes.
Cetus Corp
US 4235871 · 1980
How to Encapsulate Active Materials in Lipid Bubbles Efficiently
This patent describes a method for trapping biologically active substances inside tiny, multi-layered fat bubbles called liposomes, using a specific water-in-oil emulsion and gel-forming process to improve how much material gets captured.
Individual
Semantically similar
You might also find these interesting
US 12438891 · 2025 · Cisco Technology
How Multiple AI Models Detect Unusual Behavior on Computer Networks
US 20220012637 · Nokia Technologies Oy
Training AI Models Together with Unlabeled Data Using a Teacher
US 12518214 · 2026 · Nant Holdings IP
Training AI on Private Data Without Seeing It
US 11915111 · 2024 · CAE
How Training Systems Share AI Knowledge Without Sharing Private Student Data
More to explore
More in AI & Machine Learning
US 10452978 · 2019 · Google LLC
How AI Models Understand Language Using 'Attention'
US 6523026 · 2003 · Huntsman International LLC
How Computers Find Hidden Connections Between Different Fields of Knowledge
US 11615208 · 2023 · Capital One Services LLC
How Cloud Systems Automatically Create and Train AI Data Models
US 10402750 · 2019 · Facebook Inc
How Facebook Uses Deep Learning to Predict What You Might Like
New to patents?
Common Questions
Frequently Asked Questions
What does Improving AI Predictions by Factoring in How Much You Trust Each Model cover?
This patent describes a method for making an AI system more accurate by combining several individual AI models and giving more weight to the ones considered more trustworthy.
Who owns patent US 20230044102?
This patent is owned by Noblis.
When does this patent expire?
This patent is expected to expire on December 21, 2041, when the invention enters the public domain.
What is patent US 20230044102 cited by?
This patent has been cited by 31 later patents that build on its ideas.
What problem does this patent solve?
As AI models become more complex and are used in critical applications like healthcare or finance, ensuring their reliability and accuracy is crucial. This patent offers a way to build more robust AI systems by explicitly incorporating a measure of trust into how multiple models collaborate. This can lead to more dependable predictions, especially when individual models might have varying levels of quality or applicability to different data.
What does this patent NOT cover?
Does not cover single machine learning models that do not combine predictions from multiple individual models.
Patent monitoring