# How AI Learns Better by Understanding Connections > This patent describes a method for artificial intelligence to learn more effectively by first creating a 'structure graph' that maps relationships in its world, then using a 'graph neural network' to make smarter decisions based on that map. - **Patent:** US 20230037632 - **Original title:** Reinforcement learning method and apparatus - **Owner:** Huawei Technologies Co - **Status:** Active - **Times cited:** 11 - **Field:** ai_ml, software, robotics, telecommunications, gaming ## What it does This patent describes a method for an 'intelligent agent' (like an AI program) to learn how to make decisions in an 'environment' (like a game or a robot's world). It starts by creating a 'structure graph' (Claim 1), which is like a map showing how different parts of the environment or the agent itself are connected. This map is learned from past interactions (Claim 2). The AI then uses this structure graph, along with what's currently happening in its environment, as input to its 'policy function' (Claim 1). This policy function, which is a 'graph neural network' (Claim 1), decides what action the AI should take. After taking an action, the AI gets feedback (a 'next state' and 'reward data'), which it uses to train itself and get better at making decisions (Claim 1). For example, in a robot control scenario (Claim 6), the robot might learn the physical connections between its parts or objects in its path, using this map to plan smoother movements. ## What it does NOT cover - Does not cover reinforcement learning methods where the policy function is not a graph neural network. - Does not cover methods that do not explicitly obtain and use a 'structure graph' as an input to the policy function. - Does not cover reinforcement learning where the structure graph is not learned from historical interaction data. - Does not cover methods that do not involve training the agent through reinforcement learning based on reward data. - Does not cover scenarios where the historical interaction data is not filtered to eliminate the impact of the agent's own actions (Claims 3, 4). ## The clever bit The novelty lies in explicitly learning the underlying 'structure graph' of the environment or agent and then directly feeding this relational information into a 'graph neural network' that acts as the AI's decision-making 'policy function' during reinforcement learning. This allows the AI to make decisions that are more aware of the connections and dependencies within its world. ## Real-world examples 1. Robot control systems learning to navigate complex spaces 2. AI agents in gaming environments with intricate object interactions 3. Optimization of engineering parameters for multi-cell base stations 4. Autonomous systems needing to understand object relationships ## Why it matters Understanding the relationships between different parts of an environment or an AI agent can be crucial for complex decision-making. This patent aims to make reinforcement learning more efficient and effective by explicitly modeling these relationships using graph structures. This approach could lead to more robust and adaptable AI systems in various fields, from controlling robots to optimizing communication networks. ## Frequently asked questions ### What does How AI Learns Better by Understanding Connections cover? This patent describes a method for artificial intelligence to learn more effectively by first creating a 'structure graph' that maps relationships in its world, then using a 'graph neural network' to make smarter decisions based on that map. ### Who owns patent US 20230037632? This patent is owned by Huawei Technologies Co. ### When does this patent expire? This patent is expected to expire on October 17, 2042, when the invention enters the public domain. ### What is patent US 20230037632 cited by? This patent has been cited by 11 later patents that build on its ideas. ### What problem does this patent solve? Understanding the relationships between different parts of an environment or an AI agent can be crucial for complex decision-making. This patent aims to make reinforcement learning more efficient and effective by explicitly modeling these relationships using graph structures. This approach could lead to more robust and adaptable AI systems in various fields, from controlling robots to optimizing communication networks. ### What does this patent NOT cover? Does not cover reinforcement learning methods where the policy function is not a graph neural network. **Full plain-English explainer:** https://patentbrief.org/patent/us/20230037632/reinforcement-learning-method-and-apparatus **Original patent:** https://patents.google.com/patent/US20230037632 --- _Source: PatentBrief — https://patentbrief.org. Patent facts are from public records; the plain-English explanation is PatentBrief's._ ## Related patents Semantically similar inventions in the PatentBrief corpus: - [Using AI to Manage Wireless Network Resources Smarter](https://patentbrief.org/patent/us/12245052/reinforcement-learning-rl-and-graph-neural-network-gnn-based-resource-management) — Intel's 2025 patent on using a hierarchy of AI models to manage wireless network resources, making them more efficient by learning from network measurements and rewards. - [A Computer Chip for Training Many AI Agents at Once](https://patentbrief.org/patent/us/9754221/processor-for-implementing-reinforcement-learning-operations) — This patent describes a specialized computer processor designed to quickly train multiple artificial intelligence agents simultaneously using a unique set of instructions, speeding up how AI learns. - [How AI Models Learn Complex Tasks by Breaking Them Down](https://patentbrief.org/patent/us/11120365/for-hierarchical-decomposition-deep-reinforcement-learning-for-an-artificial-int) — This patent describes a method for training artificial intelligence models to solve big problems by splitting them into smaller, manageable steps and learning each step separately before combining them. - [How AI Agents Learn to Pick the Best Future Actions](https://patentbrief.org/patent/us/10282665/action-selection-with-a-reward-estimator-applied-to-machine-learning) — A method for an AI agent to predict which actions will yield the highest rewards by analyzing past experiences and refining its decision-making model. - [Training Robot AI Models Faster Using Smart Simulations](https://patentbrief.org/patent/us/11836577/reinforcement-learning-model-training-through-simulation) — This patent describes a cloud service that helps train artificial intelligence models for robots by running simulations, even suggesting improvements to the AI's learning rules before starting.