How AI Learns to Manage Computer Applications Using Digital Simulators
This patent describes a system that creates and trains digital simulators, which then teach other AI programs how to automatically manage and optimize complex computer applications in a controlled virtual environment.
Original patent title: “Simulator-training for automated reinforcement-learning-based application-managers”
This patent describes a system that creates and trains digital simulators, which then teach other AI programs how to automatically manage and optimize complex computer applications in a controlled virtual environment. Granted to VMware in 2022 with 23 claims and 1 forward citation, and it is expected to expire in 2039.
Coverage
What does this patent actually cover?
The patent describes a "simulation manager" that generates and trains specialized simulators (ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1). These simulators learn how a computer system behaves when an automated AI application manager takes actions. The simulator uses two main machine learning models: one predicts the next "state" (condition) of the system given a current state and an action, and another predicts a "reward" (outcome) for that state (Claim 3). To train the simulator, it receives real-world data, including actions, current states, and next states (Claim 5). It then repeatedly compares its predictions to this real data, calculating a "difference metric" (like a weighted error, Claim 6), and uses this feedback to adjust its internal models to become more accurate. For example, a human expert could use this system to configure a simulator to mimic a cloud server environment, then use that simulator to train an AI to automatically scale resources for a web application based on user traffic.
The gap
What does this patent NOT cover?
- It does not cover the automated reinforcement-learning-based application manager itself, only the system that trains it using simulators.
- It does not cover simulators that do not learn both a state-transition function (predicting next state) and a reward function (predicting reward) (ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 3).
- It does not cover training methods that do not use a "difference metric" computed from simulator-generated and training-data state transitions to adjust simulator parameters (ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 5).
- It does not cover systems where human experts directly manage the applications, only where they provide input for simulator configuration (ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1).
- It does not cover simulators that are not trained using data collected from an actual computing environment controlled by an automated reinforcement-learning-based application manager (ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 5).
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
Key facts
What made this novel
The patent's noveltynoveltyThe requirement that an invention be different from anything publicly known before its priority date.Read more → lies in its method for training the simulator itself, specifically by using weighted differences between simulated and real-world state transitions to refine the simulator's machine learning models. This feedback mechanism, combined with human expert input for model initialization, allows for more accurate and efficient simulator training.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
Cloud resource optimization for virtual machines
Automated scaling of microservices in a Kubernetes cluster
AI-driven network traffic management
Optimizing database performance in a data center
VMware's Aria Operations for Applications
Why it matters
The bigger picture
This technology is crucial for developing robust AI systems that manage complex computing environments, especially in cloud computing. Training AI directly in live production systems can be risky, expensive, and slow. By using accurate simulators, companies can rapidly test and refine AI application managers in a safe, virtual space before deploying them to real-world systems. This approach helps ensure stability and efficiency for critical software infrastructure.
Filed
July 22, 2019
Granted
February 1, 2022
Market context
Who's building on this
Companies in this space
VMware, the assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →, is a major player in cloud infrastructure and virtualization, and this patent aligns with their focus on automated operations and AI-driven management solutions like VMware Aria. Other major cloud providers such as Amazon Web Services, Google Cloud, and Microsoft Azure also invest heavily in AI-driven resource management and optimization. Startups focusing on AIOps (Artificial Intelligence for IT Operations) and cloud cost optimization would also be in this space.
Market impact
This patent supports the shift towards more autonomous and self-optimizing cloud infrastructure. It enables the development of AI systems that can manage complex application deployments with minimal human intervention, leading to increased efficiency, reduced operational costs, and improved reliability. This technology helps accelerate the adoption of AIOps solutions, making it easier for companies to leverage AI for critical IT operations.
Claim 1 — Plain English
What this patent covers
The patent describes a "simulation manager" that generates and trains specialized simulators (Claim 1). These simulators learn how a computer system behaves when an automated AI application manager takes actions. The simulator uses two main machine learning models: one predicts the next "state" (condition) of the system given a current state and an action, and another predicts a "reward" (outcome) for that state (Claim 3). To train the simulator, it receives real-world data, including actions, current states, and next states (Claim 5). It then repeatedly compares its predictions to this real data, calculating a "difference metric" (like a weighted error, Claim 6), and uses this feedback to adjust its internal models to become more accurate. For example, a human expert could use this system to configure a simulator to mimic a cloud server environment, then use that simulator to train an AI to automatically scale resources for a web application based on user traffic.
The clever bit
The patent's novelty lies in its method for training the simulator itself, specifically by using weighted differences between simulated and real-world state transitions to refine the simulator's machine learning models. This feedback mechanism, combined with human expert input for model initialization, allows for more accurate and efficient simulator training.
What it does not cover
- It does not cover the automated reinforcement-learning-based application manager itself, only the system that trains it using simulators.
- It does not cover simulators that do not learn both a state-transition function (predicting next state) and a reward function (predicting reward) (Claim 3).
- It does not cover training methods that do not use a "difference metric" computed from simulator-generated and training-data state transitions to adjust simulator parameters (Claim 5).
- It does not cover systems where human experts directly manage the applications, only where they provide input for simulator configuration (Claim 1).
- It does not cover simulators that are not trained using data collected from an actual computing environment controlled by an automated reinforcement-learning-based application manager (Claim 5).
Patent timeline
Application submitted to the patent office
Application published, typically 18 months after filing
Patent officially issued
Patent enters public domain
PatentBrief Score
Impact Score
Moderate
Citation count
6/40
Early citations
Claim breadth
15/20
Broad claimsclaimsThe numbered statements at the end of a patent that legally define what the inventor owns.Read more →
Recency
20/20
Granted within 5 years
Assignee scale
0/20
Independent or smaller assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$75K – $240K
Midpoint $150K · 12.9 yr remaining · industry ×1.6
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
Claim text not yet imported for this patent
The original legal language
Original claims
23 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
Stephen, N. M. G., Nag, D., Wang, D., Yankov, Y., & Burk, G. T. (2022). How AI Learns to Manage Computer Applications Using Digital Simulators (U.S. Patent No. 11,238,372). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/11238372/simulator-training-for-automated-reinforcement-learning-based-application-manage
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US11238372"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4965188 · 1990
How to Make Many Copies of a DNA Piece with Heat
This patent describes the Polymerase Chain Reaction (PCR) method, a technique to make millions of copies of a specific DNA segment using a heat-resistant enzyme and repeated temperature changes.
Cetus Corp
US 4235871 · 1980
How to Encapsulate Active Materials in Lipid Bubbles Efficiently
This patent describes a method for trapping biologically active substances inside tiny, multi-layered fat bubbles called liposomes, using a specific water-in-oil emulsion and gel-forming process to improve how much material gets captured.
Individual
Semantically similar
You might also find these interesting
US 11429762 · 2022 · Amazon Technologies
How Computers Train AI Models Using Separate Virtual Simulations
US 11836577 · 2023 · Amazon Technologies
Training Robot AI Models Faster Using Smart Simulations
US 11615208 · 2023 · Capital One Services LLC
How Cloud Systems Automatically Create and Train AI Data Models
US 20250371349 · Intel
How AI Learns to Run Faster on Specific Computer Chips
More to explore
More in Software & Internet
US 4405829 · 1983 · Massachusetts Institute of Technology
How RSA Public-Key Encryption Keeps Digital Messages Secret
US 6285999 · 2001 · Leland Stanford Junior University
How Websites Get Ranked by Importance
US 5960411 · 1999 · Amazon com Inc
How Amazon's One-Click Ordering Works for Online Purchases
US 7669123 · 2010 · Facebook Inc
Displaying Friends' Activities in a Social Network Feed
New to patents?
Common Questions
Frequently Asked Questions
What does How AI Learns to Manage Computer Applications Using Digital Simulators cover?
This patent describes a system that creates and trains digital simulators, which then teach other AI programs how to automatically manage and optimize complex computer applications in a controlled virtual environment.
Who owns patent US 11238372?
VMware owns this patent, granted in 2022.
When does this patent expire?
This patent is expected to expire on July 22, 2039, when the invention enters the public domain.
What is patent US 11238372 cited by?
This patent has been cited by 1 later patents that build on its ideas.
What problem does this patent solve?
This technology is crucial for developing robust AI systems that manage complex computing environments, especially in cloud computing. Training AI directly in live production systems can be risky, expensive, and slow. By using accurate simulators, companies can rapidly test and refine AI application managers in a safe, virtual space before deploying them to real-world systems. This approach helps ensure stability and efficiency for critical software infrastructure.
What does this patent NOT cover?
It does not cover the automated reinforcement-learning-based application manager itself, only the system that trains it using simulators.
Patent monitoring