A Computer Chip for Training Many AI Agents at Once
This patent describes a specialized computer processor designed to quickly train multiple artificial intelligence agents simultaneously using a unique set of instructions, speeding up how AI learns.
Original patent title: “Processor for implementing reinforcement learning operations”
This patent describes a specialized computer processor designed to quickly train multiple artificial intelligence agents simultaneously using a unique set of instructions, speeding up how AI learns. Granted to Alphaics in 2017 with 22 claims and 53 forward citations, and it is expected to expire in 2037.
Coverage
What does this patent actually cover?
The system uses a "first processor" to set up AI "reinforcement learning agents" and their "environments," assigning each a unique ID (ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1). A "first memory module" stores special instructions, part of an "application-domain specific instruction set (ASI)," which include these agent or environment IDs as "operands" (Claim 1). A "complex instruction fetch and decode (CISFD) unit" then decodes these instructions, creating "threads" that also carry the IDs (Claim 1). A "second processor" with multiple "cores" executes these threads in parallel, applying instructions to the correct agents or environments (Claim 1). This second processor determines actions, "state-value functions," "Q-values," and "reward values" for the agents. These values are stored in a "second memory module" and then used to train a "neural network" via a "neural network data path" to improve the AI's learning (Claim 1). For example, a system could train hundreds of robotic agents to navigate different simulated environments simultaneously using these specialized instructions.
The gap
What does this patent NOT cover?
- General-purpose computer processors running reinforcement learning algorithms without specialized instruction sets.
- Reinforcement learning systems that do not use agent or environment IDs as operands within their instructions.
- Hardware architectures that do not separate the creation of agents/environments from the execution of learning operations into distinct processors.
- Reinforcement learning implementations that do not communicate with a neural network via dedicated data paths for approximating reward or state-value functions.
- Systems where multiple agents are processed sequentially rather than in parallel using "Single Instruction Multiple Agents (SIMA)" type instructions.
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
Key facts
What made this novel
The core innovation is the "Single Instruction Multiple Agents (SIMA)" concept, where a single instruction can be applied simultaneously to many reinforcement learning agents interacting with their environments. This, combined with a specialized processor architecture and instruction set, allows for highly parallel and efficient training of AI.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
Google's DeepMind AlphaGo and AlphaStar training systems
NVIDIA's specialized AI accelerators for robotics simulation
Robotics training platforms
Autonomous vehicle simulation environments
Why it matters
The bigger picture
As artificial intelligence becomes more complex, training AI models, especially through reinforcement learning, requires immense computing power. This patent addresses this challenge by proposing dedicated hardware and instruction sets. This approach can significantly speed up the development and deployment of AI systems by making the learning process more efficient.
Filed
March 9, 2017
Granted
September 5, 2017
Market context
Who's building on this
Companies in this space
Companies like NVIDIA, Intel (with its Habana Labs), and Google (with its TPUs) are actively developing specialized processors and architectures for AI training and inference, including those for reinforcement learning. Alphaics, the original assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →, continues to develop AI hardware, focusing on accelerating AI computations.
Market impact
This type of specialized hardware aims to accelerate the training phase of AI models, which is a significant bottleneck in AI development. It enables faster iteration and deployment of complex AI systems, potentially leading to more sophisticated AI applications in fields like robotics, autonomous systems, and large-scale simulations. It contributes to the broader trend of custom silicon for AI, shifting away from general-purpose CPUs.
Claim 1 — Plain English
What this patent covers
The system uses a "first processor" to set up AI "reinforcement learning agents" and their "environments," assigning each a unique ID (Claim 1). A "first memory module" stores special instructions, part of an "application-domain specific instruction set (ASI)," which include these agent or environment IDs as "operands" (Claim 1). A "complex instruction fetch and decode (CISFD) unit" then decodes these instructions, creating "threads" that also carry the IDs (Claim 1). A "second processor" with multiple "cores" executes these threads in parallel, applying instructions to the correct agents or environments (Claim 1). This second processor determines actions, "state-value functions," "Q-values," and "reward values" for the agents. These values are stored in a "second memory module" and then used to train a "neural network" via a "neural network data path" to improve the AI's learning (Claim 1). For example, a system could train hundreds of robotic agents to navigate different simulated environments simultaneously using these specialized instructions.
The clever bit
The core innovation is the "Single Instruction Multiple Agents (SIMA)" concept, where a single instruction can be applied simultaneously to many reinforcement learning agents interacting with their environments. This, combined with a specialized processor architecture and instruction set, allows for highly parallel and efficient training of AI.
What it does not cover
- General-purpose computer processors running reinforcement learning algorithms without specialized instruction sets.
- Reinforcement learning systems that do not use agent or environment IDs as operands within their instructions.
- Hardware architectures that do not separate the creation of agents/environments from the execution of learning operations into distinct processors.
- Reinforcement learning implementations that do not communicate with a neural network via dedicated data paths for approximating reward or state-value functions.
- Systems where multiple agents are processed sequentially rather than in parallel using "Single Instruction Multiple Agents (SIMA)" type instructions.
Patent timeline
Application submitted to the patent office
Application published, typically 18 months after filing
Patent officially issued
Patent enters public domain
PatentBrief Score
Impact Score
Strong
Citation count
35/40
Highly cited
Claim breadth
15/20
Broad claimsclaimsThe numbered statements at the end of a patent that legally define what the inventor owns.Read more →
Recency
10/20
Granted 5–10 years ago
Assignee scale
0/20
Independent or smaller assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$312K – $998K
Midpoint $624K · 10.5 yr remaining · industry ×1.6
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
Claim text not yet imported for this patent
The original legal language
Original claims
22 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
Nagaraja, N. (2017). A Computer Chip for Training Many AI Agents at Once (U.S. Patent No. 9,754,221). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/9754221/processor-for-implementing-reinforcement-learning-operations
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US9754221"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4965188 · 1990
How to Make Many Copies of a DNA Piece with Heat
This patent describes the Polymerase Chain Reaction (PCR) method, a technique to make millions of copies of a specific DNA segment using a heat-resistant enzyme and repeated temperature changes.
Cetus Corp
US 4235871 · 1980
How to Encapsulate Active Materials in Lipid Bubbles Efficiently
This patent describes a method for trapping biologically active substances inside tiny, multi-layered fat bubbles called liposomes, using a specific water-in-oil emulsion and gel-forming process to improve how much material gets captured.
Individual
More to explore
More in AI & Machine Learning
US 10452978 · 2019 · Google LLC
How AI Models Understand Language Using 'Attention'
US 6523026 · 2003 · Huntsman International LLC
How Computers Find Hidden Connections Between Different Fields of Knowledge
US 11615208 · 2023 · Capital One Services LLC
How Cloud Systems Automatically Create and Train AI Data Models
US 10402750 · 2019 · Facebook Inc
How Facebook Uses Deep Learning to Predict What You Might Like
New to patents?
Common Questions
Frequently Asked Questions
What does A Computer Chip for Training Many AI Agents at Once cover?
This patent describes a specialized computer processor designed to quickly train multiple artificial intelligence agents simultaneously using a unique set of instructions, speeding up how AI learns.
Who owns patent US 9754221?
Alphaics owns this patent, granted in 2017.
When does this patent expire?
This patent is expected to expire on March 9, 2037, when the invention enters the public domain.
What is patent US 9754221 cited by?
This patent has been cited by 53 later patents that build on its ideas.
What problem does this patent solve?
As artificial intelligence becomes more complex, training AI models, especially through reinforcement learning, requires immense computing power. This patent addresses this challenge by proposing dedicated hardware and instruction sets. This approach can significantly speed up the development and deployment of AI systems by making the learning process more efficient.
What does this patent NOT cover?
General-purpose computer processors running reinforcement learning algorithms without specialized instruction sets.
Patent monitoring