Securely Analyzing Mixed Data with Distributed AI and Privacy

This patent describes a method for securely analyzing different types of data, like images and text, across multiple computers without collecting all the data in one place, while also protecting privacy.

What it covers

This patent outlines a method for analyzing diverse data, known as multimodal data, using a distributed approach called federated learning (S1). It sets up a system where multiple computers, called local edge nodes, can work together on data without sharing the raw information. Before participating, each node undergoes a 'multiple authentication mechanism' (S2), which includes a lightweight trusted verification using a random forest model and a calculation of its 'local model reputation' based on its F1_score and error rate. The system then performs 'multimodal data fusion' and 'multimodal data classification' on the combined dataset using a 'generalized multimodal data feature fusion model' that relies on a 'multi-head attention mechanism' (S3). This mechanism involves extracting features (e.g., 3D-CNN for images, OpenSmile for audio, Word2Vec for text), splicing them, calculating correlation scores, and using these scores to weight and sum representations for fusion and dimensionality reduction. Finally, an 'adaptive perturbation mechanism' (S4) adds noise dynamically to protect privacy, using cyclic correlation analysis and differential privacy with Gaussian noise.

What it doesn't cover

• —Does not cover data mining systems that collect all raw data into a central location before processing.
• —Does not cover systems that rely on only a single method for verifying the trustworthiness of participating data nodes.
• —Does not cover analysis of data that consists of only one type, such as just images or just audio.
• —Does not cover privacy protection methods that add noise without dynamically adjusting it based on data correlations and differential privacy.
• —Does not cover data fusion models that do not specifically use a multi-head attention mechanism for combining different data types.

The clever bit

The clever bit is combining federated learning with a robust, multi-layered security and privacy framework specifically designed for multimodal data. This includes not just authenticating nodes, but also evaluating their 'reputation' and dynamically adding noise to protect privacy based on data correlations, ensuring both collaboration and data confidentiality.

Why it matters

This technology is important because it allows powerful AI models to learn from sensitive data, like medical records, without compromising patient privacy. By keeping data localized and only sharing model updates, it addresses critical concerns in industries with strict data regulations. This approach can accelerate the development of AI applications in fields where data sharing is otherwise difficult or impossible.

Real-world examples

• 1.Medical diagnostic systems analyzing patient images, audio, and text notes from different hospitals.
• 2.Smart city applications processing sensor data, video feeds, and text reports from various city departments.
• 3.Industrial IoT platforms combining machine sensor data, maintenance logs, and operator voice commands.
• 4.Personalized health monitoring apps analyzing wearable data, user input, and environmental factors.