How AI Learns to Test Software in the Cloud
This patent describes a cloud system where a single AI model learns to test many live software applications at once by trying actions, getting feedback, and improving itself.
Original patent title: “Cloud architecture for reinforcement learning”
This patent describes a cloud system where a single AI model learns to test many live software applications at once by trying actions, getting feedback, and improving itself. Granted to Microsoft Technology Licensing in 2024 with 23 claims and 5 forward citations, and it is expected to expire in 2042.
Coverage
What does this patent actually cover?
This technology provides a way for a single artificial intelligence (AI) model, specifically a reinforcement learning model, to automatically test multiple versions of a software application running live in a cloud environment. The AI model first selects an action to take in the user interface of a software instance, such as clicking a button (ClaimclaimA numbered sentence at the end of a patent that legally defines what the inventor owns. The most important section.Read more → 1). It then causes that action to be performed and receives a 'reward' based on the outcome, like whether a task was completed (Claim 1, Claim 8). This process is repeated for other software instances, potentially running on different devices (Claim 5, Claim 10). The AI model then uses these actions and rewards from all instances to update and improve its testing strategy (Claim 1). For example, the AI could be testing a new mobile app, learning which sequence of taps and swipes leads to a successful login or exposes a bug.
The gap
What does this patent NOT cover?
- Does not cover software testing performed solely by human users.
- Does not cover testing in simulated or offline environments, as it specifies 'live environments'.
- Does not cover testing where the AI does not interact with a graphical 'user interface' or 'interface element'.
- Does not cover using a separate reinforcement learning model for each individual software instance being tested.
- Does not cover testing methods that do not involve determining a 'reward' to train the learning model.
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
Key facts
What made this novel
The noveltynoveltyThe requirement that an invention be different from anything publicly known before its priority date.Read more → lies in using a *single* reinforcement learning model to interact with and learn from *multiple live software environments* simultaneously in a distributed cloud setting. This allows the AI to gather diverse experiences and improve its testing capabilities much faster than if it were testing one instance at a time.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
Automated UI testing for web applications
AI-driven quality assurance for mobile apps
Testing new features in enterprise software across many user configurations
Continuous integration/continuous deployment (CI/CD) pipelines
Why it matters
The bigger picture
Automated software testing is crucial for quickly releasing high-quality applications. This patent describes a system that allows a single AI to efficiently test many versions of software simultaneously, which can significantly speed up the development cycle. By leveraging cloud computing, it enables large-scale, continuous testing that would be impractical or impossible with traditional methods.
Filed
June 30, 2022
Granted
June 4, 2024
Market context
Who's building on this
Companies in this space
Microsoft Technology Licensing LLC, the assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →, continues to develop cloud-based AI solutions for software development and testing. Other major cloud providers like Amazon Web Services (AWS) and Google Cloud also offer services and tools that integrate AI and machine learning into their software development lifecycles, often including automated testing capabilities. Specialized startups in the AI testing space are also exploring similar architectures.
Market impact
This technology contributes to a shift in how software is developed and maintained, moving towards more autonomous and efficient quality assurance. It enables companies to release software updates more frequently with higher confidence in their stability. By allowing a single AI to learn from many instances, it reduces the need for extensive manual testing, potentially lowering operational costs and accelerating time-to-market for new features and products across various industries.
Claim 1 — Plain English
What this patent covers
This technology provides a way for a single artificial intelligence (AI) model, specifically a reinforcement learning model, to automatically test multiple versions of a software application running live in a cloud environment. The AI model first selects an action to take in the user interface of a software instance, such as clicking a button (Claim 1). It then causes that action to be performed and receives a 'reward' based on the outcome, like whether a task was completed (Claim 1, Claim 8). This process is repeated for other software instances, potentially running on different devices (Claim 5, Claim 10). The AI model then uses these actions and rewards from all instances to update and improve its testing strategy (Claim 1). For example, the AI could be testing a new mobile app, learning which sequence of taps and swipes leads to a successful login or exposes a bug.
The clever bit
The novelty lies in using a *single* reinforcement learning model to interact with and learn from *multiple live software environments* simultaneously in a distributed cloud setting. This allows the AI to gather diverse experiences and improve its testing capabilities much faster than if it were testing one instance at a time.
What it does not cover
- Does not cover software testing performed solely by human users.
- Does not cover testing in simulated or offline environments, as it specifies 'live environments'.
- Does not cover testing where the AI does not interact with a graphical 'user interface' or 'interface element'.
- Does not cover using a separate reinforcement learning model for each individual software instance being tested.
- Does not cover testing methods that do not involve determining a 'reward' to train the learning model.
Patent timeline
Application submitted to the patent office
Application published, typically 18 months after filing
Patent officially issued
Patent enters public domain
PatentBrief Score
Impact Score
Strong
Citation count
16/40
Early citations
Claim breadth
15/20
Broad claimsclaimsThe numbered statements at the end of a patent that legally define what the inventor owns.Read more →
Recency
20/20
Granted within 5 years
Assignee scale
20/20
Major company or institution
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$94K – $300K
Midpoint $187K · 15.7 yr remaining · industry ×1.6
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
Claim text not yet imported for this patent
The original legal language
Original claims
23 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
LIM, S. K., MEZQUITA, M. M., OZGUR, L., PATEL, K. M., Chapman, T. J., Liu, X., & SPANGLER, T. P. (2024). How AI Learns to Test Software in the Cloud (U.S. Patent No. 12,001,320). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/12001320/cloud-architecture-for-reinforcement-learning
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US12001320"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4965188 · 1990
How to Make Many Copies of a DNA Piece with Heat
This patent describes the Polymerase Chain Reaction (PCR) method, a technique to make millions of copies of a specific DNA segment using a heat-resistant enzyme and repeated temperature changes.
Cetus Corp
US 4235871 · 1980
How to Encapsulate Active Materials in Lipid Bubbles Efficiently
This patent describes a method for trapping biologically active substances inside tiny, multi-layered fat bubbles called liposomes, using a specific water-in-oil emulsion and gel-forming process to improve how much material gets captured.
Individual
Semantically similar
You might also find these interesting
US 11238372 · 2022 · VMware
How AI Learns to Manage Computer Applications Using Digital Simulators
US 10963313 · 2021 · VMware
How Computer Systems Learn to Manage Themselves and Fix Their Goals
US 11836577 · 2023 · Amazon Technologies
Training Robot AI Models Faster Using Smart Simulations
US 11429762 · 2022 · Amazon Technologies
How Computers Train AI Models Using Separate Virtual Simulations
More to explore
More in Software & Internet
US 4405829 · 1983 · Massachusetts Institute of Technology
How RSA Public-Key Encryption Keeps Digital Messages Secret
US 6285999 · 2001 · Leland Stanford Junior University
How Websites Get Ranked by Importance
US 5960411 · 1999 · Amazon com Inc
How Amazon's One-Click Ordering Works for Online Purchases
US 7669123 · 2010 · Facebook Inc
Displaying Friends' Activities in a Social Network Feed
New to patents?
Common Questions
Frequently Asked Questions
What does How AI Learns to Test Software in the Cloud cover?
This patent describes a cloud system where a single AI model learns to test many live software applications at once by trying actions, getting feedback, and improving itself.
Who owns patent US 12001320?
Microsoft Technology Licensing owns this patent, granted in 2024.
When does this patent expire?
This patent is expected to expire on June 30, 2042, when the invention enters the public domain.
What is patent US 12001320 cited by?
This patent has been cited by 5 later patents that build on its ideas.
What problem does this patent solve?
Automated software testing is crucial for quickly releasing high-quality applications. This patent describes a system that allows a single AI to efficiently test many versions of software simultaneously, which can significantly speed up the development cycle. By leveraging cloud computing, it enables large-scale, continuous testing that would be impractical or impossible with traditional methods.
What does this patent NOT cover?
Does not cover software testing performed solely by human users.
Same assignee
More from Microsoft Technology Licensing
Patent monitoring