How to Make AI Run Faster on Smaller Computer Chips

A method to shrink complex AI models by converting their high-precision math into simpler, faster formats that run efficiently on mobile devices.

What it covers

This patent describes a way to compress artificial intelligence models, specifically neural networks, so they can run on hardware with limited power, like smartphones. It works by converting 'floating point' numbers—which are very precise but computationally expensive—into 'fixed point' numbers, which are much simpler for a processor to handle. The core mechanism involves analyzing the statistical 'moments' (like the average or spread) of the data flowing through the network. By shifting these values to create a 'zero-mean distribution' and adjusting the math functions accordingly, the system ensures the model stays accurate even after it has been simplified.

What it doesn't cover

• —Does not cover general neural network training methods that do not involve quantization.
• —Does not cover hardware-specific circuit designs for performing multiplication or addition.
• —Does not cover techniques that use retraining or fine-tuning to recover accuracy after quantization.
• —Does not cover non-neural network machine learning models like support vector machines.

The clever bit

Instead of just rounding numbers, the patent shifts the entire distribution of data to be centered around zero, which minimizes the mathematical errors introduced when you switch from high-precision to low-precision math.

Why it matters

As AI models grew larger, they became too heavy for mobile processors to run in real-time. This patent provides a standardized way for companies like Qualcomm to ensure their mobile chips can handle advanced AI tasks, such as image recognition or voice processing, without draining the battery or overheating the device.

Real-world examples

• 1.Qualcomm Snapdragon mobile processors
• 2.On-device AI image processing
• 3.Mobile voice assistants
• 4.Edge computing AI accelerators