How Computers Train AI Models Using Separate Virtual Simulations
This patent describes a system where one virtual computer runs simulations of a system, like a robot, and another virtual computer uses the simulation data to teach an AI model how to make better decisions.
Original patent title: “Simulation orchestration for training reinforcement learning models”
This patent describes a system where one virtual computer runs simulations of a system, like a robot, and another virtual computer uses the simulation data to teach an AI model how to make better decisions. Granted to Amazon Technologies in 2022 with 24 claims and 13 forward citations, and it is expected to expire in 2038.
Coverage
What does this patent actually cover?
The patent outlines a method for training a reinforcement learning model by orchestrating two virtual compute nodes. A 'simulation workflow manager' (from the abstractabstractA short summary at the front of the patent describing the invention. Not legally binding.Read more →) configures a first virtual computer with a training application and a second virtual computer with a simulation application (claimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1). The second computer runs a simulation of a system, such as a 'robotic device' (claim 1), which performs actions chosen by the model. Data from this simulation, including the action taken, the resulting new state, and a reward value (claim 3), is then sent to the first virtual computer. The training application on the first computer uses this data to improve the model. This process can repeat, with the updated model being sent back to the simulation for further refinement (claim 4).
The gap
What does this patent NOT cover?
- Training AI models without using a separate, dedicated simulation environment.
- Simulations that do not involve a 'robotic device' performing actions in the simulation environment (claimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1).
- Training AI models where the model does not randomize the selection of actions within the simulation (claimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1).
- Training AI models where the simulation and the model training occur on the same single compute node.
- Simulations performed for purposes other than training a reinforcement learning model for system optimization.
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
Key facts
What made this novel
The noveltynoveltyThe requirement that an invention be different from anything publicly known before its priority date.Read more → lies in the orchestrated separation of the computationally intensive simulation from the model training onto distinct, configurable virtual compute nodes. This allows for specialized scaling and optimization of each part of the reinforcement learning loop, with iterative data exchange between them.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
AWS RoboMaker
Autonomous vehicle simulation platforms
Industrial robot training systems
Drone navigation AI training
Cloud-based machine learning platforms
Why it matters
The bigger picture
This approach allows for efficient and scalable training of complex AI models, especially for robotics or autonomous systems, without needing expensive physical hardware for initial training. It enables rapid iteration and testing of AI behaviors in a safe, virtual environment. The separation of simulation and training tasks allows each component to be optimized and scaled independently, speeding up development.
Filed
November 27, 2018
Granted
August 30, 2022
Market context
Who's building on this
Companies in this space
Amazon, the assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →, actively builds on this technology through services like AWS RoboMaker, which provides a cloud-based simulation service for robotics. Other major cloud providers, such as Google Cloud and Microsoft Azure, also offer similar distributed machine learning and simulation platforms. Companies developing autonomous vehicles, industrial robots, and advanced AI systems frequently leverage such architectures for efficient model training.
Market impact
This type of distributed simulation architecture has become a standard approach for training complex reinforcement learning models, particularly for physical systems. It has enabled faster development cycles for robotics and autonomous systems by reducing reliance on physical prototypes and allowing parallel experimentation. This has led to more robust and capable AI models being deployed in real-world applications, accelerating innovation in fields like logistics, manufacturing, and transportation.
Claim 1 — Plain English
What this patent covers
The patent outlines a method for training a reinforcement learning model by orchestrating two virtual compute nodes. A 'simulation workflow manager' (from the abstract) configures a first virtual computer with a training application and a second virtual computer with a simulation application (claim 1). The second computer runs a simulation of a system, such as a 'robotic device' (claim 1), which performs actions chosen by the model. Data from this simulation, including the action taken, the resulting new state, and a reward value (claim 3), is then sent to the first virtual computer. The training application on the first computer uses this data to improve the model. This process can repeat, with the updated model being sent back to the simulation for further refinement (claim 4).
The clever bit
The novelty lies in the orchestrated separation of the computationally intensive simulation from the model training onto distinct, configurable virtual compute nodes. This allows for specialized scaling and optimization of each part of the reinforcement learning loop, with iterative data exchange between them.
What it does not cover
- Training AI models without using a separate, dedicated simulation environment.
- Simulations that do not involve a 'robotic device' performing actions in the simulation environment (claim 1).
- Training AI models where the model does not randomize the selection of actions within the simulation (claim 1).
- Training AI models where the simulation and the model training occur on the same single compute node.
- Simulations performed for purposes other than training a reinforcement learning model for system optimization.
Patent timeline
Application submitted to the patent office
Application published, typically 18 months after filing
Patent officially issued
Patent enters public domain
PatentBrief Score
Impact Score
Strong
Citation count
23/40
Moderately cited
Claim breadth
16/20
Broad claimsclaimsThe numbered statements at the end of a patent that legally define what the inventor owns.Read more →
Recency
20/20
Granted within 5 years
Assignee scale
20/20
Major company or institution
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$150K – $479K
Midpoint $300K · 12.3 yr remaining · industry ×1.6
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
Claim text not yet imported for this patent
The original legal language
Original claims
24 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
Dirac, L. P., Gene, S., Sun, E. L., Wentzel, M. C. D. C., Townsend, B. J., Kumar, P. R., Balaji, B., & Kasaragod, S. M. (2022). How Computers Train AI Models Using Separate Virtual Simulations (U.S. Patent No. 11,429,762). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/11429762/simulation-orchestration-for-training-reinforcement-learning-models
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US11429762"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4965188 · 1990
How to Make Many Copies of a DNA Piece with Heat
This patent describes the Polymerase Chain Reaction (PCR) method, a technique to make millions of copies of a specific DNA segment using a heat-resistant enzyme and repeated temperature changes.
Cetus Corp
US 4235871 · 1980
How to Encapsulate Active Materials in Lipid Bubbles Efficiently
This patent describes a method for trapping biologically active substances inside tiny, multi-layered fat bubbles called liposomes, using a specific water-in-oil emulsion and gel-forming process to improve how much material gets captured.
Individual
More to explore
More in Software & Internet
US 4405829 · 1983 · Massachusetts Institute of Technology
How RSA Public-Key Encryption Keeps Digital Messages Secret
US 6285999 · 2001 · Leland Stanford Junior University
How Websites Get Ranked by Importance
US 5960411 · 1999 · Amazon com Inc
How Amazon's One-Click Ordering Works for Online Purchases
US 7669123 · 2010 · Facebook Inc
Displaying Friends' Activities in a Social Network Feed
New to patents?
Common Questions
Frequently Asked Questions
What does How Computers Train AI Models Using Separate Virtual Simulations cover?
This patent describes a system where one virtual computer runs simulations of a system, like a robot, and another virtual computer uses the simulation data to teach an AI model how to make better decisions.
Who owns patent US 11429762?
Amazon Technologies owns this patent, granted in 2022.
When does this patent expire?
This patent is expected to expire on November 27, 2038, when the invention enters the public domain.
What is patent US 11429762 cited by?
This patent has been cited by 13 later patents that build on its ideas.
What problem does this patent solve?
This approach allows for efficient and scalable training of complex AI models, especially for robotics or autonomous systems, without needing expensive physical hardware for initial training. It enables rapid iteration and testing of AI behaviors in a safe, virtual environment. The separation of simulation and training tasks allows each component to be optimized and scaled independently, speeding up development.
What does this patent NOT cover?
Training AI models without using a separate, dedicated simulation environment.
Same assignee
More from Amazon Technologies
Patent monitoring