How AI Models Learn Complex Tasks by Breaking Them Down

This patent describes a method for training artificial intelligence models to solve big problems by splitting them into smaller, manageable steps and learning each step separately before combining them.

What it covers

The patent details an Artificial Intelligence (AI) engine with an instructor module and a learner module (Claim 1). The instructor module uses a "hierarchical-decomposition reinforcement learning technique" to break a "complex task" into multiple "individual sub-tasks" (Claim 1). Each sub-task becomes a "concept node" within a "hierarchical graph" that forms part of the AI model (Claim 1). The learner module then trains the AI objects corresponding to these individual sub-tasks, often performing this training "in parallel at the same time" (Claim 1). Crucially, the AI engine uses specific "reward functions" for each individual sub-task and then separate reward functions for the overall "end solution" of the complex task (Claim 1). This approach, combining parallel training and focused reward functions, is designed to "speed up an overall training duration" compared to training the entire complex task with a single algorithm (Claim 1). For example, teaching a robot to prepare a meal could be decomposed into sub-tasks like "identify ingredients," "chop vegetables," and "cook food," each trained with its own reward, before learning how to integrate them for the complete meal.

What it doesn't cover

• —Does not cover AI training that uses only a single, monolithic algorithm for a complex task without any decomposition into sub-tasks.
• —Does not cover AI training methods that do not employ separate reward functions for individual sub-tasks and the overall complex task's solution.
• —Does not cover AI training where the individual sub-tasks are not represented as "concept nodes" within a "hierarchical graph" structure.
• —Does not cover AI training where the individual sub-tasks are not trained, at least initially, in parallel.
• —Does not cover AI models where user input is not used to automatically partition individual sub-tasks into concept nodes.

The clever bit

The novelty lies in systematically combining hierarchical decomposition with reinforcement learning, specifically by using parallel training for individual sub-tasks and distinct reward functions at different levels of the hierarchy. This allows complex AI problems to be broken down into manageable, efficiently trainable parts, significantly speeding up the overall learning process.

Why it matters

Training AI models for complex, multi-step tasks can be incredibly time-consuming and computationally expensive. This patent offers a structured approach to make that process more efficient and scalable. By breaking down problems and training components in parallel, it helps overcome a significant bottleneck in developing advanced AI applications. This method is crucial for creating AI that can handle real-world challenges requiring sequential decision-making.

Real-world examples

• 1.Robotics for complex assembly lines or household chores
• 2.Autonomous driving systems learning navigation and specific maneuvers
• 3.AI agents in video games developing multi-stage strategies
• 4.Natural language processing models for multi-step text generation or summarization