{
  "patent_number": "US 12299406",
  "country": "US",
  "title": "How AI Checks Documents for Policy Rules and Compliance",
  "original_title": "Large language model artificial intelligence text evaluation system",
  "summary": "This patent describes a system where an AI model helps evaluate parts of a document against specific rules or policies given in plain language, then identifies problems or suggests fixes to ensure compliance.",
  "what_it_does": "The system first takes a document and divides it into smaller \"text portions.\" It then compares these portions against specific rules, called \"criteria,\" which are provided in everyday language (natural language) (Claim 1). A processor selects a subset of the most relevant text portions, often by determining relevance scores (Claim 1, 5). Next, it creates special questions, or \"criteria evaluation prompts,\" for a large language model (LLM) by combining the natural language criteria and the selected text (Claim 1). The LLM then generates new text by following the instructions in these prompts (Claim 1). This newly generated text is then evaluated to find any \"noncompliance\" with a \"designated policy\" (Claim 1). Finally, the system sends a message to a user, identifying the noncompliance (Claim 1) or even suggesting how to revise the document to become compliant (Claim 3). For example, a legal department could use this to automatically check if a contract draft adheres to specific company guidelines by feeding the guidelines as criteria and the contract as the document.",
  "what_it_does_not_cover": [
    "Evaluating an entire document without first selecting a subset of relevant text portions based on criteria (Claim 1 requires selecting a subset).",
    "Systems that do not use a generative language model to create \"novel text\" as part of the evaluation process (Claim 1 specifies determining novel text generated by a generative language model).",
    "Identifying noncompliance without evaluating the *novel text* generated by the LLM (Claim 1 states \"identifying an instance of noncompliance... by evaluating the novel text\").",
    "Systems that only provide a relevance score for text portions without also identifying policy noncompliance based on LLM-generated evaluation (Claim 1 requires identifying noncompliance and transmitting a response).",
    "Evaluating text against criteria that are *not* specified in natural language (Claim 1 specifies \"one or more criteria specified in natural language\")."
  ],
  "filed": "2024-04-19",
  "granted": "2025-05-13",
  "expires": "2044-04-19",
  "status": "active",
  "holder": "Casetext",
  "holder_url": "https://patentbrief.org/company/casetext",
  "inventors": [
    {
      "name": "Walter DeFoor",
      "url": "https://patentbrief.org/inventor/walter-defoor"
    },
    {
      "name": "Pablo Arredondo",
      "url": "https://patentbrief.org/inventor/pablo-arredondo"
    },
    {
      "name": "Ryan Walker",
      "url": "https://patentbrief.org/inventor/ryan-walker"
    },
    {
      "name": "Alan deLevie",
      "url": "https://patentbrief.org/inventor/alan-delevie"
    },
    {
      "name": "Javed Qadrud-Din",
      "url": "https://patentbrief.org/inventor/javed-qadrud-din"
    },
    {
      "name": "Brian O'Kelly",
      "url": "https://patentbrief.org/inventor/brian-okelly"
    },
    {
      "name": "Ethan Blake",
      "url": "https://patentbrief.org/inventor/ethan-blake"
    }
  ],
  "times_cited": 0,
  "tags": [
    "software",
    "ai_ml",
    "legal_tech",
    "telecommunications",
    "fintech"
  ],
  "abstract": "Relevance scores may be determined based on text included in a document. The text may be divided into a text portions, with the relevance scores being determined based on a comparison of a text portion of the plurality of text portions with a criterion specified in natural language. A subset of the plurality of text portions may be selected based on the plurality of relevance scores, with each of the subset of the plurality of text portions having a relevance score surpassing a threshold. A criteria evaluation prompt may be sent to a remote text generation modeling system via a communication interface. The criteria evaluation prompts may include an instruction to evaluate one or more of the subset of text portions against the criterion.",
  "url": "https://patentbrief.org/patent/us/12299406/large-language-model-artificial-intelligence-text-evaluation-system",
  "markdown_url": "https://patentbrief.org/patent/us/12299406/large-language-model-artificial-intelligence-text-evaluation-system/md",
  "google_patents_url": "https://patents.google.com/patent/US12299406",
  "relatedPatents": []
}