How AI Models Learn Complex Tasks by Breaking Them Down
This patent describes a method for training artificial intelligence models to solve big problems by splitting them into smaller, manageable steps and learning each step separately before combining them.
Original patent title: “For hierarchical decomposition deep reinforcement learning for an artificial intelligence model”
This patent describes a method for training artificial intelligence models to solve big problems by splitting them into smaller, manageable steps and learning each step separately before combining them. Granted to Microsoft Technology Licensing in 2021 with 23 claims and 5 forward citations, and it is expected to expire in 2038.
Coverage
What does this patent actually cover?
The patent details an Artificial Intelligence (AI) engine with an instructor module and a learner module (ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1). The instructor module uses a "hierarchical-decomposition reinforcement learning technique" to break a "complex task" into multiple "individual sub-tasks" (Claim 1). Each sub-task becomes a "concept node" within a "hierarchical graph" that forms part of the AI model (Claim 1). The learner module then trains the AI objects corresponding to these individual sub-tasks, often performing this training "in parallel at the same time" (Claim 1). Crucially, the AI engine uses specific "reward functions" for each individual sub-task and then separate reward functions for the overall "end solution" of the complex task (Claim 1). This approach, combining parallel training and focused reward functions, is designed to "speed up an overall training duration" compared to training the entire complex task with a single algorithm (Claim 1). For example, teaching a robot to prepare a meal could be decomposed into sub-tasks like "identify ingredients," "chop vegetables," and "cook food," each trained with its own reward, before learning how to integrate them for the complete meal.
The gap
What does this patent NOT cover?
- Does not cover AI training that uses only a single, monolithic algorithm for a complex task without any decomposition into sub-tasks.
- Does not cover AI training methods that do not employ separate reward functions for individual sub-tasks and the overall complex task's solution.
- Does not cover AI training where the individual sub-tasks are not represented as "concept nodes" within a "hierarchical graph" structure.
- Does not cover AI training where the individual sub-tasks are not trained, at least initially, in parallel.
- Does not cover AI models where user input is not used to automatically partition individual sub-tasks into concept nodes.
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
Key facts
What made this novel
The noveltynoveltyThe requirement that an invention be different from anything publicly known before its priority date.Read more → lies in systematically combining hierarchical decomposition with reinforcement learning, specifically by using parallel training for individual sub-tasks and distinct reward functions at different levels of the hierarchy. This allows complex AI problems to be broken down into manageable, efficiently trainable parts, significantly speeding up the overall learning process.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
Robotics for complex assembly lines or household chores
Autonomous driving systems learning navigation and specific maneuvers
AI agents in video games developing multi-stage strategies
Natural language processing models for multi-step text generation or summarization
Why it matters
The bigger picture
Training AI models for complex, multi-step tasks can be incredibly time-consuming and computationally expensive. This patent offers a structured approach to make that process more efficient and scalable. By breaking down problems and training components in parallel, it helps overcome a significant bottleneck in developing advanced AI applications. This method is crucial for creating AI that can handle real-world challenges requiring sequential decision-making.
Filed
June 14, 2018
Granted
September 14, 2021
Market context
Who's building on this
Companies in this space
Microsoft Technology Licensing LLC, the assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →, continues to develop advanced AI training methodologies across its various products and research initiatives. Other major technology companies like Google, Amazon, and Meta, along with numerous AI research labs and startups, are actively exploring and implementing hierarchical reinforcement learning techniques to build more capable and efficient AI systems for diverse applications.
Market impact
This patent contributes to the ongoing effort to make complex AI model development more practical and accessible. By providing a framework for more efficient training, it enables the creation of more sophisticated AI systems that can tackle multi-step problems. This impacts industries from robotics and autonomous systems to advanced software, by lowering the cost and time required to develop highly capable AI agents, thereby accelerating innovation and product development in these fields.
Claim 1 — Plain English
What this patent covers
The patent details an Artificial Intelligence (AI) engine with an instructor module and a learner module (Claim 1). The instructor module uses a "hierarchical-decomposition reinforcement learning technique" to break a "complex task" into multiple "individual sub-tasks" (Claim 1). Each sub-task becomes a "concept node" within a "hierarchical graph" that forms part of the AI model (Claim 1). The learner module then trains the AI objects corresponding to these individual sub-tasks, often performing this training "in parallel at the same time" (Claim 1). Crucially, the AI engine uses specific "reward functions" for each individual sub-task and then separate reward functions for the overall "end solution" of the complex task (Claim 1). This approach, combining parallel training and focused reward functions, is designed to "speed up an overall training duration" compared to training the entire complex task with a single algorithm (Claim 1). For example, teaching a robot to prepare a meal could be decomposed into sub-tasks like "identify ingredients," "chop vegetables," and "cook food," each trained with its own reward, before learning how to integrate them for the complete meal.
The clever bit
The novelty lies in systematically combining hierarchical decomposition with reinforcement learning, specifically by using parallel training for individual sub-tasks and distinct reward functions at different levels of the hierarchy. This allows complex AI problems to be broken down into manageable, efficiently trainable parts, significantly speeding up the overall learning process.
What it does not cover
- Does not cover AI training that uses only a single, monolithic algorithm for a complex task without any decomposition into sub-tasks.
- Does not cover AI training methods that do not employ separate reward functions for individual sub-tasks and the overall complex task's solution.
- Does not cover AI training where the individual sub-tasks are not represented as "concept nodes" within a "hierarchical graph" structure.
- Does not cover AI training where the individual sub-tasks are not trained, at least initially, in parallel.
- Does not cover AI models where user input is not used to automatically partition individual sub-tasks into concept nodes.
Patent timeline
Application submitted to the patent office
Application published, typically 18 months after filing
Patent officially issued
Patent enters public domain
PatentBrief Score
Impact Score
Strong
Citation count
16/40
Early citations
Claim breadth
15/20
Broad claimsclaimsThe numbered statements at the end of a patent that legally define what the inventor owns.Read more →
Recency
20/20
Granted within 5 years
Assignee scale
20/20
Major company or institution
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$78K – $250K
Midpoint $156K · 11.8 yr remaining · industry ×1.6
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
Claim text not yet imported for this patent
The original legal language
Original claims
23 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
GUDIMELLA, A., SHNAYDER, V., Story, R., Campos, M., Brown, M., Kong, R., & Shaker, M. (2021). How AI Models Learn Complex Tasks by Breaking Them Down (U.S. Patent No. 11,120,365). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/11120365/for-hierarchical-decomposition-deep-reinforcement-learning-for-an-artificial-int
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US11120365"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4965188 · 1990
How to Make Many Copies of a DNA Piece with Heat
This patent describes the Polymerase Chain Reaction (PCR) method, a technique to make millions of copies of a specific DNA segment using a heat-resistant enzyme and repeated temperature changes.
Cetus Corp
US 4235871 · 1980
How to Encapsulate Active Materials in Lipid Bubbles Efficiently
This patent describes a method for trapping biologically active substances inside tiny, multi-layered fat bubbles called liposomes, using a specific water-in-oil emulsion and gel-forming process to improve how much material gets captured.
Individual
Semantically similar
You might also find these interesting
US 11429762 · 2022 · Amazon Technologies
How Computers Train AI Models Using Separate Virtual Simulations
US 11295174 · 2022 · Royal Bank of Canada
How AI Agents Learn Faster by Predicting Opponents in Parallel
US 20250371349 · Intel
How AI Learns to Run Faster on Specific Computer Chips
US 11170293 · 2021 · Microsoft Technology Licensing LLC
How AI Systems Learn to Predict and Act Simultaneously
More to explore
More in AI & Machine Learning
US 10452978 · 2019 · Google LLC
How AI Models Understand Language Using 'Attention'
US 6523026 · 2003 · Huntsman International LLC
How Computers Find Hidden Connections Between Different Fields of Knowledge
US 11615208 · 2023 · Capital One Services LLC
How Cloud Systems Automatically Create and Train AI Data Models
US 10402750 · 2019 · Facebook Inc
How Facebook Uses Deep Learning to Predict What You Might Like
New to patents?
Common Questions
Frequently Asked Questions
What does How AI Models Learn Complex Tasks by Breaking Them Down cover?
This patent describes a method for training artificial intelligence models to solve big problems by splitting them into smaller, manageable steps and learning each step separately before combining them.
Who owns patent US 11120365?
Microsoft Technology Licensing owns this patent, granted in 2021.
When does this patent expire?
This patent is expected to expire on June 14, 2038, when the invention enters the public domain.
What is patent US 11120365 cited by?
This patent has been cited by 5 later patents that build on its ideas.
What problem does this patent solve?
Training AI models for complex, multi-step tasks can be incredibly time-consuming and computationally expensive. This patent offers a structured approach to make that process more efficient and scalable. By breaking down problems and training components in parallel, it helps overcome a significant bottleneck in developing advanced AI applications. This method is crucial for creating AI that can handle real-world challenges requiring sequential decision-making.
What does this patent NOT cover?
Does not cover AI training that uses only a single, monolithic algorithm for a complex task without any decomposition into sub-tasks.
Same assignee
More from Microsoft Technology Licensing
Patent monitoring