# How to Speed Up Neural Network Training Using Momentum > A 1988 method for training artificial neural networks that uses an 'activating variable' to speed up how quickly the network learns from its mistakes. - **Patent:** US 4914603 - **Original title:** Training neural networks - **Owner:** GTE Laboratories Inc - **Granted:** 1990 - **Status:** Public domain (expired) - **Times cited:** 29 - **Field:** ai_ml, software, semiconductors ## What it does The patent describes a way to train neural networks by splitting each neural unit into two parts. The first part handles the primary learning, while the second part tracks changes over time to create an 'activating variable.' This variable acts like momentum; it is added to the feedback signal to accelerate how quickly the network adjusts its internal variables (weights) to reach a correct solution. By comparing the network's output to a desired target and iterating through examples, the system converges on a result faster than standard methods of the time. ## What it does NOT cover - Does not cover modern deep learning architectures like Transformers or Convolutional Neural Networks. - Does not cover hardware-specific implementations like GPUs or TPUs. - Does not cover unsupervised learning techniques where no desired output is provided. - Does not cover backpropagation algorithms that do not utilize this specific two-subunit momentum mechanism. ## The clever bit The invention uses a dual-subunit structure where one subunit operates with a longer time constant than the other, effectively creating a 'memory' of past updates to accelerate future ones. ## Real-world examples 1. Early artificial neural network research software 2. Gradient descent optimization algorithms with momentum ## Why it matters This patent represents an early attempt to solve the 'slow learning' problem in artificial neural networks during the late 1980s. It predates the modern deep learning boom and highlights the historical focus on optimization techniques that remain foundational to how we train AI today, specifically the concept of momentum in gradient descent. ## Frequently asked questions ### What does How to Speed Up Neural Network Training Using Momentum cover? A 1988 method for training artificial neural networks that uses an 'activating variable' to speed up how quickly the network learns from its mistakes. ### Who owns patent US 4914603? GTE Laboratories Inc owns this patent, granted in 1990. ### When does this patent expire? This patent has expired and is now in the public domain — anyone can use the invention freely. ### What is patent US 4914603 cited by? This patent has been cited by 29 later patents that build on its ideas. ### What problem does this patent solve? This patent represents an early attempt to solve the 'slow learning' problem in artificial neural networks during the late 1980s. It predates the modern deep learning boom and highlights the historical focus on optimization techniques that remain foundational to how we train AI today, specifically the concept of momentum in gradient descent. ### What does this patent NOT cover? Does not cover modern deep learning architectures like Transformers or Convolutional Neural Networks. **Full plain-English explainer:** https://patentbrief.org/patent/us/4914603/training-neural-networks **Original patent:** https://patents.google.com/patent/US4914603 --- _Source: PatentBrief — https://patentbrief.org. Patent facts are from public records; the plain-English explanation is PatentBrief's._ ## Related patents Semantically similar inventions in the PatentBrief corpus: - [Making AI Smarter by Focusing on Unsure 'Nodes'](https://patentbrief.org/patent/us/12423586/training-nodes-of-a-neural-network-to-be-decisive) — This 2025 patent from D5AI LLC describes a way to train AI models more effectively by boosting the learning signal for 'nodes' that aren't making clear decisions on data. - [How a Chip Uses Memory to Speed Up AI Calculations](https://patentbrief.org/patent/us/11741188/hardware-accelerated-discretized-neural-network) — This patent describes a specialized computer chip that uses non-volatile memory and analog signals to quickly perform calculations for artificial intelligence, especially for neural networks that need to remember past information. - [How to Fix Faulty Memory Cells in AI Chips](https://patentbrief.org/patent/us/10956815/killing-asymmetric-resistive-processing-units-for-neural-network-training) — This patent describes a system that tests individual memory cells in AI chips for uneven behavior and then permanently disables the faulty ones before the chip starts learning, making AI training more efficient. - [How to Automatically Expand Neural Networks by Adding New Nodes](https://patentbrief.org/patent/us/10832138/gpt-language-model-pre-training) — A method for growing artificial intelligence models by identifying underperforming parts of a network and adding new nodes based on the behavior of existing ones. - [How Projection Neural Networks Speed Up AI Predictions](https://patentbrief.org/patent/us/11544573/llama-large-language-model-architecture) — A method for making artificial intelligence models faster and more efficient by using fixed, non-trainable projections to simplify complex data before processing.