Training Robot AI Models Faster Using Smart Simulations
This patent describes a cloud service that helps train artificial intelligence models for robots by running simulations, even suggesting improvements to the AI's learning rules before starting.
Original patent title: “Reinforcement learning model training through simulation”
This patent describes a cloud service that helps train artificial intelligence models for robots by running simulations, even suggesting improvements to the AI's learning rules before starting. Granted to Amazon Technologies in 2023 with 23 claims, and it is expected to expire in 2038.
Key facts
Coverage
What does this patent actually cover?
This patent details a computer-implemented method where a 'simulation management service' receives code from a customer. This code defines a 'reinforcement function' for training an AI model for a system, like a robot (ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1). The service then evaluates this code and suggests ways to improve it, based on past experiences with similar code (Claim 1). After modifying the code, the service creates a 'simulation environment' and injects the improved code into a 'simulation application' for the robot (Claim 1). Finally, it performs the reinforcement learning within this simulated world. For example, the simulation might select a robot's 'state' (like its position) and 'actions' (like moving forward), then provide a 'reward value' based on how well the action performed, which helps the AI model learn and improve (Claim 2, 4).
The gap
What does this patent NOT cover?
- Does not cover training reinforcement learning models directly on physical robots without using a simulation environment.
- Does not cover systems that train AI models without first evaluating and suggesting modifications to the customer's reinforcement function code.
- Does not cover other types of machine learning, like supervised or unsupervised learning, that do not involve a reinforcement function and reward-based training.
- Does not cover a simulation system where the user's code is not injected into a pre-existing simulation application.
- Does not cover a system that doesn't use prior code or historical data to generate suggestions for modifying the reinforcement function.
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
What made this novel
The truly clever part is the 'simulation management service' automatically evaluating the customer's reinforcement function code and suggesting modifications based on prior data. This proactive optimization helps ensure the AI model learns more efficiently and effectively before the simulation even begins.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
Amazon Web Services (AWS) RoboMaker
Cloud-based robotics simulation platforms
Autonomous vehicle training simulators
Industrial automation robot training
Logistics and warehouse robot pathfinding optimization
Why it matters
The bigger picture
Training complex AI models for robots or autonomous systems is difficult and expensive in the real world. This patent matters because it provides a structured, cloud-based way to accelerate this training in a safe, virtual environment. By automatically suggesting improvements to the learning code, it helps developers create more effective AI models faster, reducing development costs and time for applications like warehouse automation or self-driving vehicles.
Filed
November 27, 2018
Granted
December 5, 2023
Market context
Who's building on this
Companies in this space
Amazon, through its AWS services, is actively building and offering cloud-based simulation platforms for robotics and other AI development. Other major cloud providers like Google Cloud and Microsoft Azure also offer similar services for training AI models in simulated environments. Robotics companies and autonomous vehicle developers are key users of such platforms to accelerate their AI development cycles.
Market impact
This patent contributes to the growing market for cloud-based AI development and simulation services. It enables companies to develop and test complex AI models, particularly for robotics and autonomous systems, more efficiently and safely than real-world testing. This approach helps lower the barrier to entry for AI development, allowing more innovation in areas like logistics, manufacturing, and autonomous navigation by providing optimized training tools.
Claim 1 — Plain English
What this patent covers
This patent details a computer-implemented method where a 'simulation management service' receives code from a customer. This code defines a 'reinforcement function' for training an AI model for a system, like a robot (Claim 1). The service then evaluates this code and suggests ways to improve it, based on past experiences with similar code (Claim 1). After modifying the code, the service creates a 'simulation environment' and injects the improved code into a 'simulation application' for the robot (Claim 1). Finally, it performs the reinforcement learning within this simulated world. For example, the simulation might select a robot's 'state' (like its position) and 'actions' (like moving forward), then provide a 'reward value' based on how well the action performed, which helps the AI model learn and improve (Claim 2, 4).
The clever bit
The truly clever part is the 'simulation management service' automatically evaluating the customer's reinforcement function code and suggesting modifications based on prior data. This proactive optimization helps ensure the AI model learns more efficiently and effectively before the simulation even begins.
What it does not cover
- Does not cover training reinforcement learning models directly on physical robots without using a simulation environment.
- Does not cover systems that train AI models without first evaluating and suggesting modifications to the customer's reinforcement function code.
- Does not cover other types of machine learning, like supervised or unsupervised learning, that do not involve a reinforcement function and reward-based training.
- Does not cover a simulation system where the user's code is not injected into a pre-existing simulation application.
- Does not cover a system that doesn't use prior code or historical data to generate suggestions for modifying the reinforcement function.
Patent timeline
Application submitted to the patent office
Application published, typically 18 months after filing
Patent officially issued
Patent enters public domain
PatentBrief Score
Impact Score
Moderate
Citation count
0/40
No citations yet
Claim breadth
15/20
Broad claimsclaimsThe numbered statements at the end of a patent that legally define what the inventor owns.Read more →
Recency
20/20
Granted within 5 years
Assignee scale
20/20
Major company or institution
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$26K – $84K
Midpoint $53K · 12.4 yr remaining · industry ×0.9
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
The original legal language
Original claims
23 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
Dirac, L. P., Sun, E. L., Wentzel, M. C. D. C., Genc, S., Balaji, B., & Kasaragod, S. M. (2023). Training Robot AI Models Faster Using Smart Simulations (U.S. Patent No. 11,836,577). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/11836577/reinforcement-learning-model-training-through-simulation
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US11836577"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4965188 · 1990
How to Make Many Copies of a DNA Piece with Heat
This patent describes the Polymerase Chain Reaction (PCR) method, a technique to make millions of copies of a specific DNA segment using a heat-resistant enzyme and repeated temperature changes.
Cetus Corp
US 4235871 · 1980
How to Encapsulate Active Materials in Lipid Bubbles Efficiently
This patent describes a method for trapping biologically active substances inside tiny, multi-layered fat bubbles called liposomes, using a specific water-in-oil emulsion and gel-forming process to improve how much material gets captured.
Individual
Semantically similar
You might also find these interesting
US 11436484 · 2022 · Nvidia Corp
Nvidia's Method for Training Self-Driving Car AI in Simulations
US 10282665 · 2019 · Sony Corp
How AI Agents Learn to Pick the Best Future Actions
US 12518214 · 2026 · Nant Holdings IP
Training AI on Private Data Without Seeing It
US 10607134 · 2020
How AI Learns to Control Game Characters Based on Their Surroundings
More to explore
More in AI & Machine Learning
US 10452978 · 2019 · Google LLC
How AI Models Understand Language Using 'Attention'
US 6523026 · 2003 · Huntsman International LLC
How Computers Find Hidden Connections Between Different Fields of Knowledge
US 11615208 · 2023 · Capital One Services LLC
How Cloud Systems Automatically Create and Train AI Data Models
US 10402750 · 2019 · Facebook Inc
How Facebook Uses Deep Learning to Predict What You Might Like
New to patents?
Common Questions
Frequently Asked Questions
What does Training Robot AI Models Faster Using Smart Simulations cover?
This patent describes a cloud service that helps train artificial intelligence models for robots by running simulations, even suggesting improvements to the AI's learning rules before starting.
Who owns patent US 11836577?
Amazon Technologies owns this patent, granted in 2023.
When does this patent expire?
This patent is expected to expire on November 27, 2038, when the invention enters the public domain.
What problem does this patent solve?
Training complex AI models for robots or autonomous systems is difficult and expensive in the real world. This patent matters because it provides a structured, cloud-based way to accelerate this training in a safe, virtual environment. By automatically suggesting improvements to the learning code, it helps developers create more effective AI models faster, reducing development costs and time for applications like warehouse automation or self-driving vehicles.
What does this patent NOT cover?
Does not cover training reinforcement learning models directly on physical robots without using a simulation environment.
Patent monitoring