How AI Designs Its Own Brains: Hierarchical Neural Architecture Search

This patent describes a computer method for automatically designing neural networks by representing their structures as multi-level graphs and evolving the best designs based on how well they perform.

What it covers

The patent describes a computer-implemented method for automatically finding the best design, or "architecture," for a neural network. It does this by representing a neural network's structure as a "hierarchical set of graphs" (Claim 1). Think of these graphs as blueprints with different levels of detail. At the lowest level, the graphs use basic "primitive neural network operations" like simple calculations or connections (Claim 1, 2, 3). Higher levels build more complex operations by combining these simpler graphs. The method then creates a group of sample neural network designs, trains them, and measures how well they perform using a "fitness value" (Claim 1). It then selects the best-performing designs and can even "mutate" them, meaning it makes small changes to create new, potentially better designs, and repeats the process (Claim 8). For example, it might try different combinations of convolutional operations (Claim 3) or even no-connection operations (Claim 5) between nodes to see which arrangement yields the highest fitness.

What it doesn't cover

• —Does not cover manual design of neural network architectures by human experts.
• —Does not cover methods for determining architectures that do not use a hierarchical graph representation.
• —Does not cover methods that evaluate architectures without determining a "fitness value" (Claim 1).
• —Does not cover methods that do not involve "generating data defining a hierarchical set of graphs" (Claim 1).
• —Does not cover methods where the operations for higher-level graphs are not selected from preceding hierarchical levels (Claim 1).

The clever bit

The novelty lies in using a "hierarchical set of graphs" to represent neural network architectures, where complex operations are built recursively from simpler ones. This structured representation allows for systematic exploration and evolution of designs, making the search for optimal architectures more efficient and scalable than flat representations.

Why it matters

Designing effective neural networks is incredibly complex and time-consuming for humans. This patent addresses that challenge by automating the process, allowing computers to discover novel and efficient architectures that might be difficult for human experts to conceive. This automation can accelerate advancements in artificial intelligence, making powerful AI models more accessible and faster to develop for various applications.

Real-world examples

• 1.Automated Machine Learning (AutoML) platforms
• 2.Neural Architecture Search (NAS) tools
• 3.AI model development for computer vision
• 4.AI model development for natural language processing