How to Speed Up Neural Network Training Using Momentum
A 1988 method for training artificial neural networks that uses an 'activating variable' to speed up how quickly the network learns from its mistakes.
Original patent title: “Training neural networks”
A 1988 method for training artificial neural networks that uses an 'activating variable' to speed up how quickly the network learns from its mistakes. Granted to GTE Laboratories Inc in 1990 with 3 claims and 29 forward citations, and it is now in the public domain.
Coverage
What does this patent actually cover?
The patent describes a way to train neural networks by splitting each neural unit into two parts. The first part handles the primary learning, while the second part tracks changes over time to create an 'activating variable.' This variable acts like momentum; it is added to the feedback signal to accelerate how quickly the network adjusts its internal variables (weights) to reach a correct solution. By comparing the network's output to a desired target and iterating through examples, the system converges on a result faster than standard methods of the time.
The gap
What does this patent NOT cover?
- Does not cover modern deep learning architectures like Transformers or Convolutional Neural Networks.
- Does not cover hardware-specific implementations like GPUs or TPUs.
- Does not cover unsupervised learning techniques where no desired output is provided.
- Does not cover backpropagation algorithms that do not utilize this specific two-subunit momentum mechanism.
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
Key facts
What made this novel
The invention uses a dual-subunit structure where one subunit operates with a longer time constant than the other, effectively creating a 'memory' of past updates to accelerate future ones.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
Early artificial neural network research software
Gradient descent optimization algorithms with momentum
Why it matters
The bigger picture
This patent represents an early attempt to solve the 'slow learning' problem in artificial neural networks during the late 1980s. It predates the modern deep learning boom and highlights the historical focus on optimization techniques that remain foundational to how we train AI today, specifically the concept of momentum in gradient descent.
Filed
December 14, 1988
Granted
April 3, 1990
Market context
Who's building on this
Companies in this space
Modern AI research labs like OpenAI, Google DeepMind, and Meta AI build on the core concept of momentum-based optimization, though they use far more sophisticated mathematical variants than those described in this 1988 filing.
Market impact
This patent helped formalize the use of momentum in neural network training, a technique that became a standard component of optimization algorithms used to train the massive models powering today's AI industry.
Claim 1 — Plain English
What this patent covers
The patent describes a way to train neural networks by splitting each neural unit into two parts. The first part handles the primary learning, while the second part tracks changes over time to create an 'activating variable.' This variable acts like momentum; it is added to the feedback signal to accelerate how quickly the network adjusts its internal variables (weights) to reach a correct solution. By comparing the network's output to a desired target and iterating through examples, the system converges on a result faster than standard methods of the time.
The clever bit
The invention uses a dual-subunit structure where one subunit operates with a longer time constant than the other, effectively creating a 'memory' of past updates to accelerate future ones.
What it does not cover
- Does not cover modern deep learning architectures like Transformers or Convolutional Neural Networks.
- Does not cover hardware-specific implementations like GPUs or TPUs.
- Does not cover unsupervised learning techniques where no desired output is provided.
- Does not cover backpropagation algorithms that do not utilize this specific two-subunit momentum mechanism.
Patent timeline
Application submitted to the patent office
Application published, typically 18 months after filing
Patent officially issued
Patent enters public domain
This patent is in the public domain
See the Freedom to Build guide — what is free to use, what is not, and how to cite this patent.
PatentBrief Score
Impact Score
Early stage
Citation count
29/40
Moderately cited
Claim breadth
2/20
Narrow claimsclaimsThe numbered statements at the end of a patent that legally define what the inventor owns.Read more →
Recency
0/20
Older than 20 years
Assignee scale
0/20
Independent or smaller assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$17K – $55K
Midpoint $35K · expired or expiring · industry ×1.6
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
Claim text not yet imported for this patent
The original legal language
Original claims
3 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
Wood, L. F. (1990). How to Speed Up Neural Network Training Using Momentum (U.S. Patent No. 4,914,603). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/4914603/training-neural-networks
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US4914603"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4965188 · 1990
How to Make Many Copies of a DNA Piece with Heat
This patent describes the Polymerase Chain Reaction (PCR) method, a technique to make millions of copies of a specific DNA segment using a heat-resistant enzyme and repeated temperature changes.
Cetus Corp
US 4235871 · 1980
How to Encapsulate Active Materials in Lipid Bubbles Efficiently
This patent describes a method for trapping biologically active substances inside tiny, multi-layered fat bubbles called liposomes, using a specific water-in-oil emulsion and gel-forming process to improve how much material gets captured.
Individual
Semantically similar
You might also find these interesting
US 12423586 · 2025 · D5AI
Making AI Smarter by Focusing on Unsure 'Nodes'
US 11741188 · 2023 · Western Digital Technologies
How a Chip Uses Memory to Speed Up AI Calculations
US 10956815 · 2021 · International Business Machines
How to Fix Faulty Memory Cells in AI Chips
US 10832138 · 2020 · Samsung Electronics Co Ltd
How to Automatically Expand Neural Networks by Adding New Nodes
More to explore
More in AI & Machine Learning
US 10452978 · 2019 · Google LLC
How AI Models Understand Language Using 'Attention'
US 6523026 · 2003 · Huntsman International LLC
How Computers Find Hidden Connections Between Different Fields of Knowledge
US 11615208 · 2023 · Capital One Services LLC
How Cloud Systems Automatically Create and Train AI Data Models
US 10402750 · 2019 · Facebook Inc
How Facebook Uses Deep Learning to Predict What You Might Like
New to patents?
Common Questions
Frequently Asked Questions
What does How to Speed Up Neural Network Training Using Momentum cover?
A 1988 method for training artificial neural networks that uses an 'activating variable' to speed up how quickly the network learns from its mistakes.
Who owns patent US 4914603?
GTE Laboratories Inc owns this patent, granted in 1990.
When does this patent expire?
This patent has expired and is now in the public domain — anyone can use the invention freely.
What is patent US 4914603 cited by?
This patent has been cited by 29 later patents that build on its ideas.
What problem does this patent solve?
This patent represents an early attempt to solve the 'slow learning' problem in artificial neural networks during the late 1980s. It predates the modern deep learning boom and highlights the historical focus on optimization techniques that remain foundational to how we train AI today, specifically the concept of momentum in gradient descent.
What does this patent NOT cover?
Does not cover modern deep learning architectures like Transformers or Convolutional Neural Networks.
Patent monitoring