# How AI Agents Learn to Pick the Best Future Actions > A method for an AI agent to predict which actions will yield the highest rewards by analyzing past experiences and refining its decision-making model. - **Patent:** US 10282665 - **Original title:** Action selection with a reward estimator applied to machine learning - **Owner:** Sony Corp - **Granted:** 2019 - **Status:** Active - **Times cited:** 0 - **Field:** ai_ml, consumer_electronics, gaming, robotics ## What it does This patent describes a system where an AI agent learns from its history to make better decisions. It records 'action history data'—which includes the state the agent was in, the action it took, and the reward it received. The system uses this data to build a 'reward estimator' that predicts how much reward a future action might generate. By comparing these predicted rewards for various possible next steps, the agent selects and executes the action with the highest estimated value. This process allows the agent to continuously improve its performance as it gathers more data. ## What it does NOT cover - Does not cover general machine learning algorithms that do not specifically use reward estimation based on action history data. - Does not cover hardware-specific implementations, as the claims focus on the logical process performed by a CPU. - Does not cover reinforcement learning methods that rely solely on trial-and-error without a basis-function-based reward estimator. ## The clever bit The system uses 'basis functions' to transform raw state and action data into 'feature amount vectors,' which allows the AI to map complex, high-dimensional experiences into a space where it can more easily calculate and predict rewards. ## Real-world examples 1. Autonomous robot navigation in warehouses 2. Non-player character (NPC) behavior in video games 3. Automated resource management in cloud computing ## Why it matters This technology is fundamental to modern autonomous systems, such as robotics and game AI, where an agent must navigate complex environments. By formalizing how an agent evaluates the potential 'reward' of its next move, Sony provides a framework for more efficient decision-making in unpredictable scenarios. It represents a shift toward more structured, data-driven behavior in automated agents. ## Frequently asked questions ### What does How AI Agents Learn to Pick the Best Future Actions cover? A method for an AI agent to predict which actions will yield the highest rewards by analyzing past experiences and refining its decision-making model. ### Who owns patent US 10282665? Sony Corp owns this patent, granted in 2019. ### When does this patent expire? This patent is expected to expire on June 12, 2035, when the invention enters the public domain. ### What problem does this patent solve? This technology is fundamental to modern autonomous systems, such as robotics and game AI, where an agent must navigate complex environments. By formalizing how an agent evaluates the potential 'reward' of its next move, Sony provides a framework for more efficient decision-making in unpredictable scenarios. It represents a shift toward more structured, data-driven behavior in automated agents. ### What does this patent NOT cover? Does not cover general machine learning algorithms that do not specifically use reward estimation based on action history data. **Full plain-English explainer:** https://patentbrief.org/patent/us/10282665/action-selection-with-a-reward-estimator-applied-to-machine-learning **Original patent:** https://patents.google.com/patent/US10282665 --- _Source: PatentBrief — https://patentbrief.org. Patent facts are from public records; the plain-English explanation is PatentBrief's._ ## Related patents Semantically similar inventions in the PatentBrief corpus: - [How AI Systems Learn to Predict and Act Simultaneously](https://patentbrief.org/patent/us/11170293/alphago-policy-and-value-networks) — A method for training AI models that combines supervised learning for prediction with reinforcement learning for decision-making in a single, coordinated system. - [How AI Learns to Control Game Characters Based on Their Surroundings](https://patentbrief.org/patent/us/10607134/artificially-intelligent-systems-devices-and-methods-for-learning-andor-using-an-avatars-circumstances-for-autonomous-avatar-operation) — A system that allows digital characters to automatically perform actions by matching their current environment to previously learned experiences stored in a database. - [Training Robot AI Models Faster Using Smart Simulations](https://patentbrief.org/patent/us/11836577/reinforcement-learning-model-training-through-simulation) — This patent describes a cloud service that helps train artificial intelligence models for robots by running simulations, even suggesting improvements to the AI's learning rules before starting. - [A Computer Chip for Training Many AI Agents at Once](https://patentbrief.org/patent/us/9754221/processor-for-implementing-reinforcement-learning-operations) — This patent describes a specialized computer processor designed to quickly train multiple artificial intelligence agents simultaneously using a unique set of instructions, speeding up how AI learns. - [How AI Learns to Manage Computer Applications Using Digital Simulators](https://patentbrief.org/patent/us/11238372/simulator-training-for-automated-reinforcement-learning-based-application-manage) — This patent describes a system that creates and trains digital simulators, which then teach other AI programs how to automatically manage and optimize complex computer applications in a controlled virtual environment.