How a Chip Uses Memory to Speed Up AI Calculations

This patent describes a specialized computer chip that uses non-volatile memory and analog signals to quickly perform calculations for artificial intelligence, especially for neural networks that need to remember past information.

What it covers

This patent describes a device and method for speeding up neural network calculations using specialized hardware. It starts by converting digital input signals into analog signals using a Digital-to-Analog Converter (DAC). These analog signals then go into non-volatile memory (NVM) weight arrays, which perform complex math called vector matrix multiplication (VMM). A key part is that each "synaptic weight element" (like a connection strength in a brain model) is represented by multiple parallel NVM cells, such as at least three parallel NVM cells (Claim 4, 13), which helps with accuracy. After the VMM, Analog-to-Digital Converters (ADCs) turn the analog results back into digital values. A "neural circuit" then processes these digital values, applying activation functions to calculate a "new memory cell state" (Claim 1, 10), which can be fed back for the next calculation cycle, useful for AI that processes sequences like language.

What it doesn't cover

• —Does not cover neural network computations performed entirely in the digital domain without analog conversion.
• —Does not cover systems where synaptic weight elements are represented by a single NVM cell, as it specifies "multiple parallel NVM cells" (Claim 1, 10).
• —Does not cover VMM computations performed using volatile memory (like DRAM or SRAM) instead of non-volatile memory (NVM) weight arrays.
• —Does not cover systems that do not convert digital input signals to analog for VMM computation.
• —Does not cover systems that do not convert analog VMM results back to digital values using ADCs.

The clever bit

The clever part is using multiple parallel non-volatile memory (NVM) cells to represent a single "synaptic weight element" and performing vector matrix multiplication (VMM) in the analog domain. This allows for higher precision and robustness in analog computing, which is typically noisy, and enables efficient in-memory computation, reducing data movement bottlenecks.

Why it matters

This technology matters because it addresses a major challenge in AI: making neural networks run faster and more efficiently, especially on devices with limited power like smartphones or IoT sensors. By performing calculations directly within memory using analog signals, it can potentially reduce the energy and time needed to move data between a processor and memory. This approach is crucial for deploying advanced AI models, such as those used in natural language processing or voice assistants, directly on edge devices without relying on constant cloud connectivity.

Real-world examples

• 1.AI accelerators for edge devices
• 2.Specialized chips for neural network inference
• 3.Smartphones with dedicated AI processing units
• 4.IoT devices performing local AI tasks
• 5.Data center AI inference engines