# How AI Agents Learn to Pick the Best Future Actions > A method for an AI agent to predict which actions will yield the highest rewards by analyzing past experiences and refining its decision-making model. - **Patent:** US 10282665 - **Original title:** Action selection with a reward estimator applied to machine learning - **Owner:** Sony Corp - **Granted:** 2019 - **Status:** Active - **Times cited:** 0 - **Field:** ai_ml, consumer_electronics, gaming, robotics ## What it does This patent describes a system where an AI agent learns from its history to make better decisions. It records 'action history data'—which includes the state the agent was in, the action it took, and the reward it received. The system uses this data to build a 'reward estimator' that predicts how much reward a future action might generate. By comparing these predicted rewards for various possible next steps, the agent selects and executes the action with the highest estimated value. This process allows the agent to continuously improve its performance as it gathers more data. ## What it does NOT cover - Does not cover general machine learning algorithms that do not specifically use reward estimation based on action history data. - Does not cover hardware-specific implementations, as the claims focus on the logical process performed by a CPU. - Does not cover reinforcement learning methods that rely solely on trial-and-error without a basis-function-based reward estimator. ## The clever bit The system uses 'basis functions' to transform raw state and action data into 'feature amount vectors,' which allows the AI to map complex, high-dimensional experiences into a space where it can more easily calculate and predict rewards. ## Real-world examples 1. Autonomous robot navigation in warehouses 2. Non-player character (NPC) behavior in video games 3. Automated resource management in cloud computing ## Why it matters This technology is fundamental to modern autonomous systems, such as robotics and game AI, where an agent must navigate complex environments. By formalizing how an agent evaluates the potential 'reward' of its next move, Sony provides a framework for more efficient decision-making in unpredictable scenarios. It represents a shift toward more structured, data-driven behavior in automated agents. ## Frequently asked questions ### What does How AI Agents Learn to Pick the Best Future Actions cover? A method for an AI agent to predict which actions will yield the highest rewards by analyzing past experiences and refining its decision-making model. ### Who owns patent US 10282665? Sony Corp owns this patent, granted in 2019. ### When does this patent expire? This patent is expected to expire on June 12, 2035, when the invention enters the public domain. ### What problem does this patent solve? This technology is fundamental to modern autonomous systems, such as robotics and game AI, where an agent must navigate complex environments. By formalizing how an agent evaluates the potential 'reward' of its next move, Sony provides a framework for more efficient decision-making in unpredictable scenarios. It represents a shift toward more structured, data-driven behavior in automated agents. ### What does this patent NOT cover? Does not cover general machine learning algorithms that do not specifically use reward estimation based on action history data. **Full plain-English explainer:** https://patentbrief.org/patent/us/10282665/action-selection-with-a-reward-estimator-applied-to-machine-learning **Original patent:** https://patents.google.com/patent/US10282665 --- _Source: PatentBrief — https://patentbrief.org. Patent facts are from public records; the plain-English explanation is PatentBrief's._ ## Related patents Semantically similar inventions in the PatentBrief corpus: - [How AI Learns to Control Game Characters Based on Their Surroundings](https://patentbrief.org/patent/us/10607134/artificially-intelligent-systems-devices-and-methods-for-learning-andor-using-an-avatars-circumstances-for-autonomous-avatar-operation) — A system that allows digital characters to automatically perform actions by matching their current environment to previously learned experiences stored in a database. - [Training Robot AI Models Faster Using Smart Simulations](https://patentbrief.org/patent/us/11836577/reinforcement-learning-model-training-through-simulation) — This patent describes a cloud service that helps train artificial intelligence models for robots by running simulations, even suggesting improvements to the AI's learning rules before starting. - [How Devices Predict Your Next Move Using Gestures and History](https://patentbrief.org/patent/us/RE46310/google-drive) — A patent for a smart interface that tracks your physical gestures and past habits to predict what you want to do next, showing custom options on a screen to save you time. - [Making AI Smarter by Focusing on Unsure 'Nodes'](https://patentbrief.org/patent/us/12423586/training-nodes-of-a-neural-network-to-be-decisive) — This 2025 patent from D5AI LLC describes a way to train AI models more effectively by boosting the learning signal for 'nodes' that aren't making clear decisions on data. - [Nvidia's Method for Training Self-Driving Car AI in Simulations](https://patentbrief.org/patent/us/11436484/alphazero) — Nvidia's 2022 patent describes how to train AI for self-driving cars by using simulated environments and virtual sensors, then matching the simulated data format to real-world sensor data for AI processing.