How AI Learns to Manage Computer Applications Using Digital Simulators

This patent describes a system that creates and trains digital simulators, which then teach other AI programs how to automatically manage and optimize complex computer applications in a controlled virtual environment.

What it covers

The patent describes a "simulation manager" that generates and trains specialized simulators (Claim 1). These simulators learn how a computer system behaves when an automated AI application manager takes actions. The simulator uses two main machine learning models: one predicts the next "state" (condition) of the system given a current state and an action, and another predicts a "reward" (outcome) for that state (Claim 3). To train the simulator, it receives real-world data, including actions, current states, and next states (Claim 5). It then repeatedly compares its predictions to this real data, calculating a "difference metric" (like a weighted error, Claim 6), and uses this feedback to adjust its internal models to become more accurate. For example, a human expert could use this system to configure a simulator to mimic a cloud server environment, then use that simulator to train an AI to automatically scale resources for a web application based on user traffic.

What it doesn't cover

• —It does not cover the automated reinforcement-learning-based application manager itself, only the system that trains it using simulators.
• —It does not cover simulators that do not learn both a state-transition function (predicting next state) and a reward function (predicting reward) (Claim 3).
• —It does not cover training methods that do not use a "difference metric" computed from simulator-generated and training-data state transitions to adjust simulator parameters (Claim 5).
• —It does not cover systems where human experts directly manage the applications, only where they provide input for simulator configuration (Claim 1).
• —It does not cover simulators that are not trained using data collected from an actual computing environment controlled by an automated reinforcement-learning-based application manager (Claim 5).

The clever bit

The patent's novelty lies in its method for training the simulator itself, specifically by using weighted differences between simulated and real-world state transitions to refine the simulator's machine learning models. This feedback mechanism, combined with human expert input for model initialization, allows for more accurate and efficient simulator training.

Why it matters

This technology is crucial for developing robust AI systems that manage complex computing environments, especially in cloud computing. Training AI directly in live production systems can be risky, expensive, and slow. By using accurate simulators, companies can rapidly test and refine AI application managers in a safe, virtual space before deploying them to real-world systems. This approach helps ensure stability and efficiency for critical software infrastructure.

Real-world examples

• 1.Cloud resource optimization for virtual machines
• 2.Automated scaling of microservices in a Kubernetes cluster
• 3.AI-driven network traffic management
• 4.Optimizing database performance in a data center
• 5.VMware's Aria Operations for Applications