How Devices Train Shared AI Models While Keeping Your Data Private

This patent describes a method for training a machine learning model across many devices, where each device keeps some parts of the model and its data private, only sharing updates for the common, global parts of the model.

What it covers

This patent outlines a method for a client device to help train a larger machine learning model without sending its private data to a central server. The model has two parts: "local parameters" unique to the client, and "global parameters" shared across all devices. The client first receives the current global parameters from a server. Then, using a "support dataset" (a part of its local data not used for global training), it figures out the best values for its own local parameters (Claim 1). After that, it uses a "query dataset" (another part of its local data) to train and update only the global parameters (Claim 1). Finally, it sends back only the changes to the global parameters, specifically designed so that the server cannot reconstruct the client's private local data or local parameters (Claim 1). For example, your phone could help improve a predictive text model by training it on your typing habits, but only send back general improvements to the shared dictionary, never your specific messages.

What it doesn't cover

• —Does not cover traditional centralized machine learning where all raw data is sent to a single server for training.
• —Does not cover federated learning systems that only have global parameters and no distinct local parameters on the client device.
• —Does not cover methods where the client computing device transmits updates to its local parameters to the server.
• —Does not cover systems where the parameter update data allows for the recovery of the local data maintained at the client device.
• —Does not cover training methods that do not partition the local data into separate "support" and "query" datasets for different training steps.

The clever bit

The novelty lies in the "partially local" approach, where a client device maintains and refines its own unique "local parameters" alongside the shared "global parameters." Critically, it only sends updates for the global parameters, ensuring that its private local data cannot be reconstructed from the transmitted information, a significant privacy enhancement over earlier federated learning methods.

Why it matters

This technology is crucial for developing powerful AI models while protecting user privacy. By allowing models to be trained on data directly on user devices, it avoids the need to collect and store sensitive information centrally. This approach helps companies like Google improve services such as predictive keyboards, voice assistants, and photo organization without ever seeing individual user data, which is increasingly important due to data privacy regulations.

Real-world examples

• 1.Predictive text suggestions on smartphones
• 2.On-device photo categorization and search
• 3.Voice assistant personalization
• 4.Personalized recommendation engines
• 5.Health monitoring applications that train on sensitive user data