# How AI Models Learn Complex Tasks by Breaking Them Down > This patent describes a method for training artificial intelligence models to solve big problems by splitting them into smaller, manageable steps and learning each step separately before combining them. - **Patent:** US 11120365 - **Original title:** For hierarchical decomposition deep reinforcement learning for an artificial intelligence model - **Owner:** Microsoft Technology Licensing - **Granted:** 2021 - **Status:** Active - **Times cited:** 5 - **Field:** ai_ml, software, telecommunications, consumer_electronics ## What it does The patent details an Artificial Intelligence (AI) engine with an instructor module and a learner module (Claim 1). The instructor module uses a "hierarchical-decomposition reinforcement learning technique" to break a "complex task" into multiple "individual sub-tasks" (Claim 1). Each sub-task becomes a "concept node" within a "hierarchical graph" that forms part of the AI model (Claim 1). The learner module then trains the AI objects corresponding to these individual sub-tasks, often performing this training "in parallel at the same time" (Claim 1). Crucially, the AI engine uses specific "reward functions" for each individual sub-task and then separate reward functions for the overall "end solution" of the complex task (Claim 1). This approach, combining parallel training and focused reward functions, is designed to "speed up an overall training duration" compared to training the entire complex task with a single algorithm (Claim 1). For example, teaching a robot to prepare a meal could be decomposed into sub-tasks like "identify ingredients," "chop vegetables," and "cook food," each trained with its own reward, before learning how to integrate them for the complete meal. ## What it does NOT cover - Does not cover AI training that uses only a single, monolithic algorithm for a complex task without any decomposition into sub-tasks. - Does not cover AI training methods that do not employ separate reward functions for individual sub-tasks and the overall complex task's solution. - Does not cover AI training where the individual sub-tasks are not represented as "concept nodes" within a "hierarchical graph" structure. - Does not cover AI training where the individual sub-tasks are not trained, at least initially, in parallel. - Does not cover AI models where user input is not used to automatically partition individual sub-tasks into concept nodes. ## The clever bit The novelty lies in systematically combining hierarchical decomposition with reinforcement learning, specifically by using parallel training for individual sub-tasks and distinct reward functions at different levels of the hierarchy. This allows complex AI problems to be broken down into manageable, efficiently trainable parts, significantly speeding up the overall learning process. ## Real-world examples 1. Robotics for complex assembly lines or household chores 2. Autonomous driving systems learning navigation and specific maneuvers 3. AI agents in video games developing multi-stage strategies 4. Natural language processing models for multi-step text generation or summarization ## Why it matters Training AI models for complex, multi-step tasks can be incredibly time-consuming and computationally expensive. This patent offers a structured approach to make that process more efficient and scalable. By breaking down problems and training components in parallel, it helps overcome a significant bottleneck in developing advanced AI applications. This method is crucial for creating AI that can handle real-world challenges requiring sequential decision-making. ## Frequently asked questions ### What does How AI Models Learn Complex Tasks by Breaking Them Down cover? This patent describes a method for training artificial intelligence models to solve big problems by splitting them into smaller, manageable steps and learning each step separately before combining them. ### Who owns patent US 11120365? Microsoft Technology Licensing owns this patent, granted in 2021. ### When does this patent expire? This patent is expected to expire on June 14, 2038, when the invention enters the public domain. ### What is patent US 11120365 cited by? This patent has been cited by 5 later patents that build on its ideas. ### What problem does this patent solve? Training AI models for complex, multi-step tasks can be incredibly time-consuming and computationally expensive. This patent offers a structured approach to make that process more efficient and scalable. By breaking down problems and training components in parallel, it helps overcome a significant bottleneck in developing advanced AI applications. This method is crucial for creating AI that can handle real-world challenges requiring sequential decision-making. ### What does this patent NOT cover? Does not cover AI training that uses only a single, monolithic algorithm for a complex task without any decomposition into sub-tasks. **Full plain-English explainer:** https://patentbrief.org/patent/us/11120365/for-hierarchical-decomposition-deep-reinforcement-learning-for-an-artificial-int **Original patent:** https://patents.google.com/patent/US11120365 --- _Source: PatentBrief — https://patentbrief.org. Patent facts are from public records; the plain-English explanation is PatentBrief's._ ## Related patents Semantically similar inventions in the PatentBrief corpus: - [How Computers Train AI Models Using Separate Virtual Simulations](https://patentbrief.org/patent/us/11429762/simulation-orchestration-for-training-reinforcement-learning-models) — This patent describes a system where one virtual computer runs simulations of a system, like a robot, and another virtual computer uses the simulation data to teach an AI model how to make better decisions. - [How AI Agents Learn Faster by Predicting Opponents in Parallel](https://patentbrief.org/patent/us/11295174/opponent-modeling-with-asynchronous-methods-in-deep-rl) — This patent describes a computer system that trains artificial intelligence agents more efficiently by using many processors to simultaneously learn from a simulated environment, while also explicitly predicting the actions of other agents. - [How AI Learns to Run Faster on Specific Computer Chips](https://patentbrief.org/patent/us/20250371349/methods-and-apparatus-for-hardware-aware-machine-learning-model-training) — This patent describes how a smart AI system, called a reinforcement learning agent, trains other AI models to run more efficiently on specific computer hardware by cleverly reducing their size without losing too much accuracy. - [How AI Systems Learn to Predict and Act Simultaneously](https://patentbrief.org/patent/us/11170293/alphago-policy-and-value-networks) — A method for training AI models that combines supervised learning for prediction with reinforcement learning for decision-making in a single, coordinated system. - [Training Robot AI Models Faster Using Smart Simulations](https://patentbrief.org/patent/us/11836577/reinforcement-learning-model-training-through-simulation) — This patent describes a cloud service that helps train artificial intelligence models for robots by running simulations, even suggesting improvements to the AI's learning rules before starting.