How AI Models Can Be More Confident in Their Document Answers
This patent describes a system for making machine learning models more accurate by adjusting how confident they are in the data they extract from documents, after the initial extraction.
Original patent title: “Enhanced machine learning model accuracy through post-hoc confidence score calibration”
This patent describes a system for making machine learning models more accurate by adjusting how confident they are in the data they extract from documents, after the initial extraction. Granted to Snowflake in 2025 with 33 claims, and it is expected to expire in 2044.
Coverage
What does this patent actually cover?
A machine learning (ML) system first receives results from an AI model, which include a confidence score and electronic documents (ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1). It then performs "document understanding" to pull out specific "data points" from these documents, like a date or an amount (Claim 1). The system connects the initial confidence score to these extracted data points. Next, it uses a "post-hoc calibration (PHC) solution set" (Claim 1) to adjust these confidence scores. This calibration step involves methods like "temperature scaling" or "polynomial regression" (Claim 3) to make sure the confidence score truly matches how accurate the AI's prediction is (Claim 1). After adjusting, the system extracts the final pieces of information, like a specific value, and outputs these values along with their newly adjusted, more reliable confidence scores (Claim 1). For example, if an AI extracts a shipping address from a contract, this patent helps ensure the AI's reported confidence level for that address is truly accurate.
The gap
What does this patent NOT cover?
- Does not cover improving ML model accuracy during the initial training or prediction phase, only after results are generated.
- Does not cover ML models that do not involve "document understanding" or extracting data points from electronic documents.
- Does not cover methods of improving ML model performance that do not involve calibrating confidence scores.
- Does not cover calibration methods that are not "post-hoc" or do not use techniques like temperature scaling, softmax top-k, or polynomial regression.
- Does not cover general model improvements that are not specifically related to aligning confidence scores with prediction accuracy.
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
Key facts
What made this novel
The clever bit is recognizing that an AI's initial confidence score might not truly reflect its accuracy, and then providing specific methods to recalibrate that confidence after the AI has already made its predictions and extracted data from documents. This ensures the reported confidence is a more honest reflection of how likely the extracted information is to be correct.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
Snowflake Cortex (Snowflake's AI services)
AI-powered invoice processing systems
Automated contract review software
Intelligent document processing platforms
Large language models used for data extraction
Why it matters
The bigger picture
Many AI models, especially large language models, can be overly confident in their predictions, even when they are wrong. This patent matters because it provides a structured way to make an AI's stated confidence level more truthful. This is crucial for applications where relying on AI-extracted information, like financial figures or legal clauses from documents, requires high accuracy and trustworthiness. By ensuring the confidence score truly reflects the likelihood of correctness, businesses can better decide when to trust AI outputs versus when human review is needed.
Filed
May 29, 2024
Granted
February 25, 2025
Market context
Who's building on this
Companies in this space
Snowflake Inc., the assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →, is actively integrating machine learning capabilities into its data cloud platform, including document processing. Other major cloud providers like Google Cloud, Microsoft Azure, and Amazon Web Services also offer AI services that could benefit from or incorporate similar confidence calibration techniques for their document understanding and large language model offerings. Startups focused on Intelligent Document Processing (IDP) also operate in this space.
Market impact
This patent contributes to increasing trust and reliability in AI-driven data extraction, particularly from unstructured documents. By providing a robust method for calibrating confidence scores, it enables businesses to automate more processes with greater assurance, potentially reducing the need for extensive human oversight in tasks like invoice processing or contract analysis. This enhances the value proposition of AI solutions in enterprise settings, making them more practical for critical business operations.
Claim 1 — Plain English
What this patent covers
A machine learning (ML) system first receives results from an AI model, which include a confidence score and electronic documents (Claim 1). It then performs "document understanding" to pull out specific "data points" from these documents, like a date or an amount (Claim 1). The system connects the initial confidence score to these extracted data points. Next, it uses a "post-hoc calibration (PHC) solution set" (Claim 1) to adjust these confidence scores. This calibration step involves methods like "temperature scaling" or "polynomial regression" (Claim 3) to make sure the confidence score truly matches how accurate the AI's prediction is (Claim 1). After adjusting, the system extracts the final pieces of information, like a specific value, and outputs these values along with their newly adjusted, more reliable confidence scores (Claim 1). For example, if an AI extracts a shipping address from a contract, this patent helps ensure the AI's reported confidence level for that address is truly accurate.
The clever bit
The clever bit is recognizing that an AI's initial confidence score might not truly reflect its accuracy, and then providing specific methods to recalibrate that confidence after the AI has already made its predictions and extracted data from documents. This ensures the reported confidence is a more honest reflection of how likely the extracted information is to be correct.
What it does not cover
- Does not cover improving ML model accuracy during the initial training or prediction phase, only after results are generated.
- Does not cover ML models that do not involve "document understanding" or extracting data points from electronic documents.
- Does not cover methods of improving ML model performance that do not involve calibrating confidence scores.
- Does not cover calibration methods that are not "post-hoc" or do not use techniques like temperature scaling, softmax top-k, or polynomial regression.
- Does not cover general model improvements that are not specifically related to aligning confidence scores with prediction accuracy.
Patent timeline
Application submitted to the patent office
Application published, typically 18 months after filing
Patent officially issued
Patent enters public domain
PatentBrief Score
Impact Score
Moderate
Citation count
0/40
No citations yet
Claim breadth
20/20
Very broad protection
Recency
20/20
Granted within 5 years
Assignee scale
0/20
Independent or smaller assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$38K – $123K
Midpoint $77K · 17.6 yr remaining · industry ×1.6
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
Claim text not yet imported for this patent
The original legal language
Original claims
33 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
Szwabe, A. (2025). How AI Models Can Be More Confident in Their Document Answers (U.S. Patent No. 12,236,201). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/12236201/enhanced-machine-learning-model-accuracy-through-post-hoc-confidence-score-calib
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US12236201"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4965188 · 1990
How to Make Many Copies of a DNA Piece with Heat
This patent describes the Polymerase Chain Reaction (PCR) method, a technique to make millions of copies of a specific DNA segment using a heat-resistant enzyme and repeated temperature changes.
Cetus Corp
US 4235871 · 1980
How to Encapsulate Active Materials in Lipid Bubbles Efficiently
This patent describes a method for trapping biologically active substances inside tiny, multi-layered fat bubbles called liposomes, using a specific water-in-oil emulsion and gel-forming process to improve how much material gets captured.
Individual
Semantically similar
You might also find these interesting
US 20230044102 · Noblis
Improving AI Predictions by Factoring in How Much You Trust Each Model
US 10599957 · 2020 · Capital One Services
How to Automatically Detect and Fix Changes in AI Model Data
US 12299406 · 2025 · Casetext
How AI Checks Documents for Policy Rules and Compliance
US 11989527 · 2024 · Unlikely Artificial Intelligence
How a Computer System Checks and Improves AI Text
More to explore
More in Software & Internet
US 4405829 · 1983 · Massachusetts Institute of Technology
How RSA Public-Key Encryption Keeps Digital Messages Secret
US 6285999 · 2001 · Leland Stanford Junior University
How Websites Get Ranked by Importance
US 5960411 · 1999 · Amazon com Inc
How Amazon's One-Click Ordering Works for Online Purchases
US 7669123 · 2010 · Facebook Inc
Displaying Friends' Activities in a Social Network Feed
New to patents?
Common Questions
Frequently Asked Questions
What does How AI Models Can Be More Confident in Their Document Answers cover?
This patent describes a system for making machine learning models more accurate by adjusting how confident they are in the data they extract from documents, after the initial extraction.
Who owns patent US 12236201?
Snowflake owns this patent, granted in 2025.
When does this patent expire?
This patent is expected to expire on May 29, 2044, when the invention enters the public domain.
What problem does this patent solve?
Many AI models, especially large language models, can be overly confident in their predictions, even when they are wrong. This patent matters because it provides a structured way to make an AI's stated confidence level more truthful. This is crucial for applications where relying on AI-extracted information, like financial figures or legal clauses from documents, requires high accuracy and trustworthiness. By ensuring the confidence score truly reflects the likelihood of correctness, businesses can better decide when to trust AI outputs versus when human review is needed.
What does this patent NOT cover?
Does not cover improving ML model accuracy during the initial training or prediction phase, only after results are generated.
Patent monitoring