How AI Checks Documents for Policy Rules and Compliance

This patent describes a system where an AI model helps evaluate parts of a document against specific rules or policies given in plain language, then identifies problems or suggests fixes to ensure compliance.

What it covers

The system first takes a document and divides it into smaller "text portions." It then compares these portions against specific rules, called "criteria," which are provided in everyday language (natural language) (Claim 1). A processor selects a subset of the most relevant text portions, often by determining relevance scores (Claim 1, 5). Next, it creates special questions, or "criteria evaluation prompts," for a large language model (LLM) by combining the natural language criteria and the selected text (Claim 1). The LLM then generates new text by following the instructions in these prompts (Claim 1). This newly generated text is then evaluated to find any "noncompliance" with a "designated policy" (Claim 1). Finally, the system sends a message to a user, identifying the noncompliance (Claim 1) or even suggesting how to revise the document to become compliant (Claim 3). For example, a legal department could use this to automatically check if a contract draft adheres to specific company guidelines by feeding the guidelines as criteria and the contract as the document.

What it doesn't cover

• —Evaluating an entire document without first selecting a subset of relevant text portions based on criteria (Claim 1 requires selecting a subset).
• —Systems that do not use a generative language model to create "novel text" as part of the evaluation process (Claim 1 specifies determining novel text generated by a generative language model).
• —Identifying noncompliance without evaluating the *novel text* generated by the LLM (Claim 1 states "identifying an instance of noncompliance... by evaluating the novel text").
• —Systems that only provide a relevance score for text portions without also identifying policy noncompliance based on LLM-generated evaluation (Claim 1 requires identifying noncompliance and transmitting a response).
• —Evaluating text against criteria that are *not* specified in natural language (Claim 1 specifies "one or more criteria specified in natural language").

The clever bit

The novelty lies in using a generative language model not just to find relevant text, but to generate novel text by executing an instruction to evaluate the selected text against criteria, and then evaluating that generated text to identify policy noncompliance. This moves beyond simple keyword matching or sentiment analysis by having the AI actively interpret and explain compliance.

Why it matters

This technology is important for automating compliance checks and document review in fields that deal with large volumes of text and complex rules. It aims to make processes faster and more accurate by leveraging advanced AI models to understand and apply nuanced policy criteria. This could significantly reduce manual effort and human error in sectors like legal, regulatory, and corporate governance.

Real-world examples

• 1.Automated legal document review platforms
• 2.Contract compliance checking software
• 3.Regulatory policy adherence verification tools
• 4.Internal policy auditing systems
• 5.Content moderation tools for platform guidelines