Training AI Models Together with Unlabeled Data Using a Teacher

This patent describes a way for multiple AI systems to learn together from data that hasn't been manually labeled, using a 'teacher' AI to create temporary labels for a 'student' AI.

What it covers

This patent describes an apparatus, or node, within a federated machine learning system. This node contains a 'federated student machine learning network' that updates its own AI model by considering the updated models from other nodes in the system (Claim 1). Crucially, it also has a 'teacher machine learning network' which receives data that has not been manually labeled. The teacher network then creates 'pseudo-labels' for this unlabeled data (Claim 1). The federated student network then uses this unlabeled data along with the teacher's pseudo-labels to perform supervised learning (Claim 1). For example, a network on a phone could learn to identify new types of objects in photos by getting rough labels from a local 'teacher' AI, while also sharing its learning with other phones to improve overall accuracy without sending private photos to a central server.

What it doesn't cover

• —Does not cover federated learning systems that rely solely on manually labeled data for training.
• —Does not cover machine learning systems where a 'student' network does not update its model based on other 'nodes' in a federated system.
• —Does not cover traditional centralized machine learning where all data is sent to one server for training.
• —Does not cover systems that use a teacher network but do not involve a federated student network.
• —Does not cover systems that only use unsupervised learning without generating pseudo-labels for supervised learning.

The clever bit

The clever part is combining federated learning, where multiple devices collaboratively train an AI model without sharing raw data, with a 'teacher-student' approach that generates its own labels for previously unlabeled data. This allows the system to learn from much more data without needing expensive human labeling or compromising privacy.

Why it matters

This technology is important for training powerful AI models while protecting user privacy. By allowing AI models to learn from unlabeled data directly on devices (like smartphones or IoT sensors) and only share model updates, it reduces the need to send sensitive raw data to a central server. This approach can also make AI training more efficient, as it reduces the massive human effort typically required to manually label vast amounts of data.

Real-world examples

• 1.AI models on smartphones learning to improve photo recognition without uploading personal photos.
• 2.IoT devices collaboratively learning to detect anomalies in sensor data.
• 3.Edge computing devices improving predictive maintenance models.
• 4.Healthcare systems training AI on patient data while maintaining privacy.