How Computers Train AI Models Using Separate Virtual Simulations

This patent describes a system where one virtual computer runs simulations of a system, like a robot, and another virtual computer uses the simulation data to teach an AI model how to make better decisions.

What it covers

The patent outlines a method for training a reinforcement learning model by orchestrating two virtual compute nodes. A 'simulation workflow manager' (from the abstract) configures a first virtual computer with a training application and a second virtual computer with a simulation application (claim 1). The second computer runs a simulation of a system, such as a 'robotic device' (claim 1), which performs actions chosen by the model. Data from this simulation, including the action taken, the resulting new state, and a reward value (claim 3), is then sent to the first virtual computer. The training application on the first computer uses this data to improve the model. This process can repeat, with the updated model being sent back to the simulation for further refinement (claim 4).

What it doesn't cover

• —Training AI models without using a separate, dedicated simulation environment.
• —Simulations that do not involve a 'robotic device' performing actions in the simulation environment (claim 1).
• —Training AI models where the model does not randomize the selection of actions within the simulation (claim 1).
• —Training AI models where the simulation and the model training occur on the same single compute node.
• —Simulations performed for purposes other than training a reinforcement learning model for system optimization.

The clever bit

The novelty lies in the orchestrated separation of the computationally intensive simulation from the model training onto distinct, configurable virtual compute nodes. This allows for specialized scaling and optimization of each part of the reinforcement learning loop, with iterative data exchange between them.

Why it matters

This approach allows for efficient and scalable training of complex AI models, especially for robotics or autonomous systems, without needing expensive physical hardware for initial training. It enables rapid iteration and testing of AI behaviors in a safe, virtual environment. The separation of simulation and training tasks allows each component to be optimized and scaled independently, speeding up development.

Real-world examples

• 1.AWS RoboMaker
• 2.Autonomous vehicle simulation platforms
• 3.Industrial robot training systems
• 4.Drone navigation AI training
• 5.Cloud-based machine learning platforms