How to Automatically Expand Neural Networks by Adding New Nodes

A method for growing artificial intelligence models by identifying underperforming parts of a network and adding new nodes based on the behavior of existing ones.

What it covers

This patent describes a way to make neural networks smarter by letting them grow dynamically. Instead of building a fixed-size model, the system monitors how often nodes 'fire' (activation frequency) and how consistently they behave (activation entropy). When the network hits a performance plateau, the processor identifies a specific node that needs help and adds a new one to that layer. The new node inherits some of its connection weights from the original node, while the rest are set to initial values, allowing the network to adapt to new data without starting training from scratch.

What it doesn't cover

• —Does not cover static neural networks that do not add nodes during or after training.
• —Does not cover methods of network expansion that rely on random weight initialization for all new nodes.
• —Does not cover pruning or shrinking techniques that remove nodes without adding new ones.
• —Does not cover hardware-specific implementations that do not use activation frequency or entropy as selection criteria.

The clever bit

The system uses 'activation entropy'—a measure of how often a node flips between active and inactive states—as a key metric to decide where to expand, rather than just looking at raw performance metrics.

Why it matters

Training large AI models is computationally expensive and time-consuming. By allowing a network to grow only where it is needed, this method offers a way to improve model accuracy while potentially saving resources compared to training massive, static models from the ground up. It represents a shift toward more efficient, adaptive machine learning architectures.

Real-world examples

• 1.Adaptive deep learning models for mobile devices
• 2.Dynamic neural network architectures in edge computing
• 3.On-device AI model optimization