How AI Learns Better by Understanding Connections
This patent describes a method for artificial intelligence to learn more effectively by first creating a 'structure graph' that maps relationships in its world, then using a 'graph neural network' to make smarter decisions based on that map.
Original patent title: “Reinforcement learning method and apparatus”
This patent describes a method for artificial intelligence to learn more effectively by first creating a 'structure graph' that maps relationships in its world, then using a 'graph neural network' to make smarter decisions based on that map. Owned by Huawei Technologies Co with 19 claims and 11 forward citations, and it is expected to expire in 2042.
Coverage
What does this patent actually cover?
This patent describes a method for an 'intelligent agent' (like an AI program) to learn how to make decisions in an 'environment' (like a game or a robot's world). It starts by creating a 'structure graph' (ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1), which is like a map showing how different parts of the environment or the agent itself are connected. This map is learned from past interactions (Claim 2). The AI then uses this structure graph, along with what's currently happening in its environment, as input to its 'policy function' (Claim 1). This policy function, which is a 'graph neural network' (Claim 1), decides what action the AI should take. After taking an action, the AI gets feedback (a 'next state' and 'reward data'), which it uses to train itself and get better at making decisions (Claim 1). For example, in a robot control scenario (Claim 6), the robot might learn the physical connections between its parts or objects in its path, using this map to plan smoother movements.
The gap
What does this patent NOT cover?
- Does not cover reinforcement learning methods where the policy function is not a graph neural network.
- Does not cover methods that do not explicitly obtain and use a 'structure graph' as an input to the policy function.
- Does not cover reinforcement learning where the structure graph is not learned from historical interaction data.
- Does not cover methods that do not involve training the agent through reinforcement learning based on reward data.
- Does not cover scenarios where the historical interaction data is not filtered to eliminate the impact of the agent's own actions (ClaimsclaimsThe numbered statements at the end of a patent that legally define what the inventor owns.Read more → 3, 4).
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
Key facts
What made this novel
The noveltynoveltyThe requirement that an invention be different from anything publicly known before its priority date.Read more → lies in explicitly learning the underlying 'structure graph' of the environment or agent and then directly feeding this relational information into a 'graph neural network' that acts as the AI's decision-making 'policy function' during reinforcement learning. This allows the AI to make decisions that are more aware of the connections and dependencies within its world.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
Robot control systems learning to navigate complex spaces
AI agents in gaming environments with intricate object interactions
Optimization of engineering parameters for multi-cell base stations
Autonomous systems needing to understand object relationships
Why it matters
The bigger picture
Understanding the relationships between different parts of an environment or an AI agent can be crucial for complex decision-making. This patent aims to make reinforcement learning more efficient and effective by explicitly modeling these relationships using graph structures. This approach could lead to more robust and adaptable AI systems in various fields, from controlling robots to optimizing communication networks.
Filed
October 17, 2022
Market context
Who's building on this
Companies in this space
Huawei Technologies Co Ltd, the assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →, is actively researching and applying advanced AI techniques, including reinforcement learning and graph neural networks, in areas like telecommunications and cloud computing. Other major technology companies and AI research institutions, such as Google DeepMind, Meta AI, and various university labs, are also exploring similar combinations of graph neural networks and reinforcement learning for complex AI tasks.
Market impact
This patent contributes to the ongoing evolution of artificial intelligence, particularly in making AI systems more capable of handling complex, relational data. While it's a relatively recent filing, the concepts it covers are central to improving AI's ability to learn and adapt in dynamic environments. Such advancements could lead to more sophisticated autonomous systems and more efficient optimization solutions across industries, potentially influencing future AI product development and research directions.
Claim 1 — Plain English
What this patent covers
This patent describes a method for an 'intelligent agent' (like an AI program) to learn how to make decisions in an 'environment' (like a game or a robot's world). It starts by creating a 'structure graph' (Claim 1), which is like a map showing how different parts of the environment or the agent itself are connected. This map is learned from past interactions (Claim 2). The AI then uses this structure graph, along with what's currently happening in its environment, as input to its 'policy function' (Claim 1). This policy function, which is a 'graph neural network' (Claim 1), decides what action the AI should take. After taking an action, the AI gets feedback (a 'next state' and 'reward data'), which it uses to train itself and get better at making decisions (Claim 1). For example, in a robot control scenario (Claim 6), the robot might learn the physical connections between its parts or objects in its path, using this map to plan smoother movements.
The clever bit
The novelty lies in explicitly learning the underlying 'structure graph' of the environment or agent and then directly feeding this relational information into a 'graph neural network' that acts as the AI's decision-making 'policy function' during reinforcement learning. This allows the AI to make decisions that are more aware of the connections and dependencies within its world.
What it does not cover
- Does not cover reinforcement learning methods where the policy function is not a graph neural network.
- Does not cover methods that do not explicitly obtain and use a 'structure graph' as an input to the policy function.
- Does not cover reinforcement learning where the structure graph is not learned from historical interaction data.
- Does not cover methods that do not involve training the agent through reinforcement learning based on reward data.
- Does not cover scenarios where the historical interaction data is not filtered to eliminate the impact of the agent's own actions (Claims 3, 4).
Patent timeline
Application submitted to the patent office
Patent enters public domain
PatentBrief Score
Impact Score
Moderate
Citation count
22/40
Moderately cited
Claim breadth
13/20
Broad claimsclaimsThe numbered statements at the end of a patent that legally define what the inventor owns.Read more →
Recency
0/20
Older than 20 years
Assignee scale
20/20
Major company or institution
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$100K – $319K
Midpoint $200K · 16.0 yr remaining · industry ×1.6
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
Claim text not yet imported for this patent
The original legal language
Original claims
19 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
CUN, W., LIU, F., & Chen, Z. How AI Learns Better by Understanding Connections (U.S. Patent No. 20,230,037,632). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/20230037632/reinforcement-learning-method-and-apparatus
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US20230037632"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4965188 · 1990
How to Make Many Copies of a DNA Piece with Heat
This patent describes the Polymerase Chain Reaction (PCR) method, a technique to make millions of copies of a specific DNA segment using a heat-resistant enzyme and repeated temperature changes.
Cetus Corp
US 4235871 · 1980
How to Encapsulate Active Materials in Lipid Bubbles Efficiently
This patent describes a method for trapping biologically active substances inside tiny, multi-layered fat bubbles called liposomes, using a specific water-in-oil emulsion and gel-forming process to improve how much material gets captured.
Individual
Semantically similar
You might also find these interesting
US 12245052 · 2025 · Intel
Using AI to Manage Wireless Network Resources Smarter
US 9754221 · 2017 · Alphaics
A Computer Chip for Training Many AI Agents at Once
US 11120365 · 2021 · Microsoft Technology Licensing
How AI Models Learn Complex Tasks by Breaking Them Down
US 10282665 · 2019 · Sony Corp
How AI Agents Learn to Pick the Best Future Actions
More to explore
More in AI & Machine Learning
US 10452978 · 2019 · Google LLC
How AI Models Understand Language Using 'Attention'
US 6523026 · 2003 · Huntsman International LLC
How Computers Find Hidden Connections Between Different Fields of Knowledge
US 11615208 · 2023 · Capital One Services LLC
How Cloud Systems Automatically Create and Train AI Data Models
US 10402750 · 2019 · Facebook Inc
How Facebook Uses Deep Learning to Predict What You Might Like
New to patents?
Common Questions
Frequently Asked Questions
What does How AI Learns Better by Understanding Connections cover?
This patent describes a method for artificial intelligence to learn more effectively by first creating a 'structure graph' that maps relationships in its world, then using a 'graph neural network' to make smarter decisions based on that map.
Who owns patent US 20230037632?
This patent is owned by Huawei Technologies Co.
When does this patent expire?
This patent is expected to expire on October 17, 2042, when the invention enters the public domain.
What is patent US 20230037632 cited by?
This patent has been cited by 11 later patents that build on its ideas.
What problem does this patent solve?
Understanding the relationships between different parts of an environment or an AI agent can be crucial for complex decision-making. This patent aims to make reinforcement learning more efficient and effective by explicitly modeling these relationships using graph structures. This approach could lead to more robust and adaptable AI systems in various fields, from controlling robots to optimizing communication networks.
What does this patent NOT cover?
Does not cover reinforcement learning methods where the policy function is not a graph neural network.
Same assignee
More from Huawei Technologies Co
Patent monitoring