# How AI Learns to Manage Computer Applications Using Digital Simulators > This patent describes a system that creates and trains digital simulators, which then teach other AI programs how to automatically manage and optimize complex computer applications in a controlled virtual environment. - **Patent:** US 11238372 - **Original title:** Simulator-training for automated reinforcement-learning-based application-managers - **Owner:** VMware - **Granted:** 2022 - **Status:** Active - **Times cited:** 1 - **Field:** software, ai_ml, telecommunications, cloud_computing, devops ## What it does The patent describes a "simulation manager" that generates and trains specialized simulators (Claim 1). These simulators learn how a computer system behaves when an automated AI application manager takes actions. The simulator uses two main machine learning models: one predicts the next "state" (condition) of the system given a current state and an action, and another predicts a "reward" (outcome) for that state (Claim 3). To train the simulator, it receives real-world data, including actions, current states, and next states (Claim 5). It then repeatedly compares its predictions to this real data, calculating a "difference metric" (like a weighted error, Claim 6), and uses this feedback to adjust its internal models to become more accurate. For example, a human expert could use this system to configure a simulator to mimic a cloud server environment, then use that simulator to train an AI to automatically scale resources for a web application based on user traffic. ## What it does NOT cover - It does not cover the automated reinforcement-learning-based application manager itself, only the system that trains it using simulators. - It does not cover simulators that do not learn both a state-transition function (predicting next state) and a reward function (predicting reward) (Claim 3). - It does not cover training methods that do not use a "difference metric" computed from simulator-generated and training-data state transitions to adjust simulator parameters (Claim 5). - It does not cover systems where human experts directly manage the applications, only where they provide input for simulator configuration (Claim 1). - It does not cover simulators that are not trained using data collected from an actual computing environment controlled by an automated reinforcement-learning-based application manager (Claim 5). ## The clever bit The patent's novelty lies in its method for training the simulator itself, specifically by using weighted differences between simulated and real-world state transitions to refine the simulator's machine learning models. This feedback mechanism, combined with human expert input for model initialization, allows for more accurate and efficient simulator training. ## Real-world examples 1. Cloud resource optimization for virtual machines 2. Automated scaling of microservices in a Kubernetes cluster 3. AI-driven network traffic management 4. Optimizing database performance in a data center 5. VMware's Aria Operations for Applications ## Why it matters This technology is crucial for developing robust AI systems that manage complex computing environments, especially in cloud computing. Training AI directly in live production systems can be risky, expensive, and slow. By using accurate simulators, companies can rapidly test and refine AI application managers in a safe, virtual space before deploying them to real-world systems. This approach helps ensure stability and efficiency for critical software infrastructure. ## Frequently asked questions ### What does How AI Learns to Manage Computer Applications Using Digital Simulators cover? This patent describes a system that creates and trains digital simulators, which then teach other AI programs how to automatically manage and optimize complex computer applications in a controlled virtual environment. ### Who owns patent US 11238372? VMware owns this patent, granted in 2022. ### When does this patent expire? This patent is expected to expire on July 22, 2039, when the invention enters the public domain. ### What is patent US 11238372 cited by? This patent has been cited by 1 later patents that build on its ideas. ### What problem does this patent solve? This technology is crucial for developing robust AI systems that manage complex computing environments, especially in cloud computing. Training AI directly in live production systems can be risky, expensive, and slow. By using accurate simulators, companies can rapidly test and refine AI application managers in a safe, virtual space before deploying them to real-world systems. This approach helps ensure stability and efficiency for critical software infrastructure. ### What does this patent NOT cover? It does not cover the automated reinforcement-learning-based application manager itself, only the system that trains it using simulators. **Full plain-English explainer:** https://patentbrief.org/patent/us/11238372/simulator-training-for-automated-reinforcement-learning-based-application-manage **Original patent:** https://patents.google.com/patent/US11238372 --- _Source: PatentBrief — https://patentbrief.org. Patent facts are from public records; the plain-English explanation is PatentBrief's._ ## Related patents Semantically similar inventions in the PatentBrief corpus: - [How Computers Train AI Models Using Separate Virtual Simulations](https://patentbrief.org/patent/us/11429762/simulation-orchestration-for-training-reinforcement-learning-models) — This patent describes a system where one virtual computer runs simulations of a system, like a robot, and another virtual computer uses the simulation data to teach an AI model how to make better decisions. - [Training Robot AI Models Faster Using Smart Simulations](https://patentbrief.org/patent/us/11836577/reinforcement-learning-model-training-through-simulation) — This patent describes a cloud service that helps train artificial intelligence models for robots by running simulations, even suggesting improvements to the AI's learning rules before starting. - [How Cloud Systems Automatically Create and Train AI Data Models](https://patentbrief.org/patent/us/11615208/dall-e-text-to-image-generation) — A cloud-based system that generates fake, privacy-safe data to train AI models, ensuring they remain accurate while protecting sensitive personal information. - [How AI Learns to Run Faster on Specific Computer Chips](https://patentbrief.org/patent/us/20250371349/methods-and-apparatus-for-hardware-aware-machine-learning-model-training) — This patent describes how a smart AI system, called a reinforcement learning agent, trains other AI models to run more efficiently on specific computer hardware by cleverly reducing their size without losing too much accuracy. - [How AI Agents Learn Faster by Predicting Opponents in Parallel](https://patentbrief.org/patent/us/11295174/opponent-modeling-with-asynchronous-methods-in-deep-rl) — This patent describes a computer system that trains artificial intelligence agents more efficiently by using many processors to simultaneously learn from a simulated environment, while also explicitly predicting the actions of other agents.