# How AI Learns to Test Software in the Cloud > This patent describes a cloud system where a single AI model learns to test many live software applications at once by trying actions, getting feedback, and improving itself. - **Patent:** US 12001320 - **Original title:** Cloud architecture for reinforcement learning - **Owner:** Microsoft Technology Licensing - **Granted:** 2024 - **Status:** Active - **Times cited:** 5 - **Field:** software, ai_ml, telecommunications, consumer_electronics ## What it does This technology provides a way for a single artificial intelligence (AI) model, specifically a reinforcement learning model, to automatically test multiple versions of a software application running live in a cloud environment. The AI model first selects an action to take in the user interface of a software instance, such as clicking a button (Claim 1). It then causes that action to be performed and receives a 'reward' based on the outcome, like whether a task was completed (Claim 1, Claim 8). This process is repeated for other software instances, potentially running on different devices (Claim 5, Claim 10). The AI model then uses these actions and rewards from all instances to update and improve its testing strategy (Claim 1). For example, the AI could be testing a new mobile app, learning which sequence of taps and swipes leads to a successful login or exposes a bug. ## What it does NOT cover - Does not cover software testing performed solely by human users. - Does not cover testing in simulated or offline environments, as it specifies 'live environments'. - Does not cover testing where the AI does not interact with a graphical 'user interface' or 'interface element'. - Does not cover using a separate reinforcement learning model for each individual software instance being tested. - Does not cover testing methods that do not involve determining a 'reward' to train the learning model. ## The clever bit The novelty lies in using a *single* reinforcement learning model to interact with and learn from *multiple live software environments* simultaneously in a distributed cloud setting. This allows the AI to gather diverse experiences and improve its testing capabilities much faster than if it were testing one instance at a time. ## Real-world examples 1. Automated UI testing for web applications 2. AI-driven quality assurance for mobile apps 3. Testing new features in enterprise software across many user configurations 4. Continuous integration/continuous deployment (CI/CD) pipelines ## Why it matters Automated software testing is crucial for quickly releasing high-quality applications. This patent describes a system that allows a single AI to efficiently test many versions of software simultaneously, which can significantly speed up the development cycle. By leveraging cloud computing, it enables large-scale, continuous testing that would be impractical or impossible with traditional methods. ## Frequently asked questions ### What does How AI Learns to Test Software in the Cloud cover? This patent describes a cloud system where a single AI model learns to test many live software applications at once by trying actions, getting feedback, and improving itself. ### Who owns patent US 12001320? Microsoft Technology Licensing owns this patent, granted in 2024. ### When does this patent expire? This patent is expected to expire on June 30, 2042, when the invention enters the public domain. ### What is patent US 12001320 cited by? This patent has been cited by 5 later patents that build on its ideas. ### What problem does this patent solve? Automated software testing is crucial for quickly releasing high-quality applications. This patent describes a system that allows a single AI to efficiently test many versions of software simultaneously, which can significantly speed up the development cycle. By leveraging cloud computing, it enables large-scale, continuous testing that would be impractical or impossible with traditional methods. ### What does this patent NOT cover? Does not cover software testing performed solely by human users. **Full plain-English explainer:** https://patentbrief.org/patent/us/12001320/cloud-architecture-for-reinforcement-learning **Original patent:** https://patents.google.com/patent/US12001320 --- _Source: PatentBrief — https://patentbrief.org. Patent facts are from public records; the plain-English explanation is PatentBrief's._ ## Related patents Semantically similar inventions in the PatentBrief corpus: - [How AI Learns to Manage Computer Applications Using Digital Simulators](https://patentbrief.org/patent/us/11238372/simulator-training-for-automated-reinforcement-learning-based-application-manage) — This patent describes a system that creates and trains digital simulators, which then teach other AI programs how to automatically manage and optimize complex computer applications in a controlled virtual environment. - [How Computer Systems Learn to Manage Themselves and Fix Their Goals](https://patentbrief.org/patent/us/10963313/automated-reinforcement-learning-based-application-manager-that-learns-and-impro) — This patent describes an automated system that uses reinforcement learning to manage computer applications and infrastructure, uniquely learning and adjusting its own goal-setting "reward function" based on how well it aligns with human expectations. - [Training Robot AI Models Faster Using Smart Simulations](https://patentbrief.org/patent/us/11836577/reinforcement-learning-model-training-through-simulation) — This patent describes a cloud service that helps train artificial intelligence models for robots by running simulations, even suggesting improvements to the AI's learning rules before starting. - [How Computers Train AI Models Using Separate Virtual Simulations](https://patentbrief.org/patent/us/11429762/simulation-orchestration-for-training-reinforcement-learning-models) — This patent describes a system where one virtual computer runs simulations of a system, like a robot, and another virtual computer uses the simulation data to teach an AI model how to make better decisions. - [A Computer Chip for Training Many AI Agents at Once](https://patentbrief.org/patent/us/9754221/processor-for-implementing-reinforcement-learning-operations) — This patent describes a specialized computer processor designed to quickly train multiple artificial intelligence agents simultaneously using a unique set of instructions, speeding up how AI learns.