How to Check if an AI Model Has Been Tampered With

This patent describes a method to verify the integrity and authorized use of an artificial intelligence model by feeding it a special digital key and comparing its response to a known good one.

What it covers

The patent outlines a method for validating and authenticating machine learning models. It works by first "providing a digital key that is associated with a particular entity" (Claim 1). This key is then "applied as input across at least a portion of the trained machine learning model to generate one or more verification outputs" (Claim 1). These outputs are then "compared to one or more known verification outputs" that were previously generated when the model was known to be uncompromised (Claim 1). If the comparison reveals a discrepancy, the system "determining... that one or more parameters of the trained machine learning model have been compromised" and then indicates this compromise (Claim 1). For example, a company providing an AI model for medical diagnosis could periodically feed it a unique digital key. If the model's response to this key changes from the expected output, it signals that the model might have been altered or misused.

What it doesn't cover

• —Does not cover detecting model compromise without using a specific digital key as input to the model itself.
• —Does not cover validation methods that only check the final output without comparing it to a known output generated by a prior application of the same key.
• —Does not cover general cybersecurity measures for AI models that do not involve this specific input-output verification process.
• —Does not cover methods that do not involve comparing outputs to *known* outputs generated by *prior* application of the digital key.

The clever bit

The novelty lies in treating the machine learning model itself as a system that can be 'fingerprinted' by its response to a unique 'digital key.' Instead of just using a key for access, it's used as an input to the model, and the model's internal or external behavior in response is checked against a known baseline to detect any changes.

Why it matters

Machine learning models are increasingly deployed in critical sectors like healthcare, finance, and autonomous systems. Ensuring their integrity and preventing unauthorized use or malicious tampering is vital for trust, security, and regulatory compliance. This patent offers a specific mechanism to detect if a deployed model has been altered, which could otherwise lead to incorrect predictions, biased outcomes, or security vulnerabilities.

Real-world examples

• 1.Software for detecting tampering in medical AI diagnostics platforms.
• 2.Systems for verifying the integrity of financial fraud detection models.
• 3.Platforms ensuring licensed use and preventing unauthorized copying of proprietary AI models.
• 4.Automated checks for AI models embedded in industrial control systems.