How AI Learns Better by Understanding Connections

This patent describes a method for artificial intelligence to learn more effectively by first creating a 'structure graph' that maps relationships in its world, then using a 'graph neural network' to make smarter decisions based on that map.

What it covers

This patent describes a method for an 'intelligent agent' (like an AI program) to learn how to make decisions in an 'environment' (like a game or a robot's world). It starts by creating a 'structure graph' (Claim 1), which is like a map showing how different parts of the environment or the agent itself are connected. This map is learned from past interactions (Claim 2). The AI then uses this structure graph, along with what's currently happening in its environment, as input to its 'policy function' (Claim 1). This policy function, which is a 'graph neural network' (Claim 1), decides what action the AI should take. After taking an action, the AI gets feedback (a 'next state' and 'reward data'), which it uses to train itself and get better at making decisions (Claim 1). For example, in a robot control scenario (Claim 6), the robot might learn the physical connections between its parts or objects in its path, using this map to plan smoother movements.

What it doesn't cover

• —Does not cover reinforcement learning methods where the policy function is not a graph neural network.
• —Does not cover methods that do not explicitly obtain and use a 'structure graph' as an input to the policy function.
• —Does not cover reinforcement learning where the structure graph is not learned from historical interaction data.
• —Does not cover methods that do not involve training the agent through reinforcement learning based on reward data.
• —Does not cover scenarios where the historical interaction data is not filtered to eliminate the impact of the agent's own actions (Claims 3, 4).

The clever bit

The novelty lies in explicitly learning the underlying 'structure graph' of the environment or agent and then directly feeding this relational information into a 'graph neural network' that acts as the AI's decision-making 'policy function' during reinforcement learning. This allows the AI to make decisions that are more aware of the connections and dependencies within its world.

Why it matters

Understanding the relationships between different parts of an environment or an AI agent can be crucial for complex decision-making. This patent aims to make reinforcement learning more efficient and effective by explicitly modeling these relationships using graph structures. This approach could lead to more robust and adaptable AI systems in various fields, from controlling robots to optimizing communication networks.

Real-world examples

• 1.Robot control systems learning to navigate complex spaces
• 2.AI agents in gaming environments with intricate object interactions
• 3.Optimization of engineering parameters for multi-cell base stations
• 4.Autonomous systems needing to understand object relationships