{
  "patent_number": "US 20230044102",
  "country": "US",
  "title": "Improving AI Predictions by Factoring in How Much You Trust Each Model",
  "original_title": "Ensemble machine learning models incorporating a model trust factor",
  "summary": "This patent describes a method for making an AI system more accurate by combining several individual AI models and giving more weight to the ones considered more trustworthy.",
  "what_it_does": "This patent describes a method for training an ensemble machine learning model, which is like a team of AI models working together. First, it receives data about how much to trust each individual AI model in the team (Claim 1a). This 'trust score' can be based on things like how sensitive the model is to bad input data or how confident its past predictions were (Claim 5). Next, it calculates an estimated prediction error for each model, but this error is adjusted by its trust score (Claim 1b). A less trusted model might have its errors penalized more. Then, it uses these adjusted error estimates to figure out a 'normalized weight' for each model (Claim 1c). Finally, it uses these normalized weights to determine the overall prediction equation for the entire ensemble, giving more influence to the more trusted models (Claim 1d). For example, if you have three AI models predicting stock prices, and one is known to be very reliable with certain types of data, this method would give that reliable model's predictions more say in the final combined forecast.",
  "what_it_does_not_cover": [
    "Does not cover single machine learning models that do not combine predictions from multiple individual models.",
    "Does not cover ensemble models that combine predictions without explicitly calculating and using a 'trust score' for each individual model.",
    "Does not cover methods where the trust score does not influence the prediction error estimate or the normalized weights of the individual models.",
    "Does not cover trust scores that are not real numbers, specifically those outside the 0.0 to 1.0 range as described in Claim 3.",
    "Does not cover ensemble methods where the output prediction equation is not determined, at least in part, by these normalized weights."
  ],
  "filed": "2021-12-21",
  "granted": null,
  "expires": "2041-12-21",
  "status": "active",
  "holder": "Noblis",
  "holder_url": "https://patentbrief.org/company/noblis",
  "inventors": [
    {
      "name": "Nathan KALKA",
      "url": "https://patentbrief.org/inventor/nathan-kalka"
    },
    {
      "name": "Janet Anderson",
      "url": "https://patentbrief.org/inventor/janet-anderson"
    },
    {
      "name": "Nicholas D. BARTLOW",
      "url": "https://patentbrief.org/inventor/nicholas-d-bartlow"
    }
  ],
  "times_cited": 31,
  "tags": [
    "ai_ml",
    "software",
    "telecommunications",
    "finance",
    "biotech",
    "automotive"
  ],
  "abstract": "Methods for improving the prediction accuracy for an ensemble machine learning model are described. In some instances, the methods comprise: (i) receiving data characterizing levels of trust in one or more machine learning models that form the ensemble machine learning model; (ii) calculating a prediction error estimate for each of the one or more machine learning models based on a trust score for that machine learning model and relative weights calculated for the data points in a training data set used to train that machine learning model; (iii) calculating a normalized weight for each of the one or more machine learning models using the prediction error estimate calculated for each; and (iv) adjusting an output prediction equation for the ensemble machine learning model, where the adjustment is based, at least in part, on the normalized weights calculated in for each of the one or more machine learning models.",
  "url": "https://patentbrief.org/patent/us/20230044102/ensemble-machine-learning-models-incorporating-a-model-trust-factor",
  "markdown_url": "https://patentbrief.org/patent/us/20230044102/ensemble-machine-learning-models-incorporating-a-model-trust-factor/md",
  "google_patents_url": "https://patents.google.com/patent/US20230044102",
  "relatedPatents": [
    {
      "patentNumber": "12438891",
      "countryCode": "US",
      "title": "How Multiple AI Models Detect Unusual Behavior on Computer Networks",
      "url": "https://patentbrief.org/patent/us/12438891/anomaly-detection-based-on-ensemble-machine-learning-model"
    },
    {
      "patentNumber": "20220012637",
      "countryCode": "US",
      "title": "Training AI Models Together with Unlabeled Data Using a Teacher",
      "url": "https://patentbrief.org/patent/us/20220012637/federated-teacher-student-machine-learning"
    },
    {
      "patentNumber": "12518214",
      "countryCode": "US",
      "title": "Training AI on Private Data Without Seeing It",
      "url": "https://patentbrief.org/patent/us/12518214/distributed-machine-learning-systems-including-generation-of-synthetic-data"
    },
    {
      "patentNumber": "11915111",
      "countryCode": "US",
      "title": "How Training Systems Share AI Knowledge Without Sharing Private Student Data",
      "url": "https://patentbrief.org/patent/us/11915111/federated-machine-learning-in-adaptive-training-systems"
    },
    {
      "patentNumber": "11544411",
      "countryCode": "US",
      "title": "How to Check if an AI Model Has Been Tampered With",
      "url": "https://patentbrief.org/patent/us/11544411/machine-learning-model-validation-and-authentication"
    }
  ]
}