# How Digital Assistants Control Apps and Ask for More Information > This patent describes how a digital assistant on a device can understand what a user wants from a natural language command, find the right app, get a step-by-step guide from another device, and then ask the user for more details on the screen to complete the task with that app. - **Patent:** US 11204787 - **Original title:** Application integration with a digital assistant - **Owner:** Apple Inc - **Granted:** 2021 - **Status:** Active - **Times cited:** 48 - **Field:** consumer_electronics, software, telecommunications, ai_ml ## What it does This patent details a system where an electronic device, like a smartphone, receives a natural-language voice command from a user. The device then figures out the user's 'intent' (what they want to do) and identifies a specific software application on the device that can perform that task. Uniquely, the device then receives a 'task flow'—a series of programmed steps—from a *second electronic device* (Claim 1). This task flow guides how the primary device interacts with the identified app. If the app needs more information to complete the task, it sends a request. The device then uses the touch-sensitive display to show a 'query' (a question) to the user, determined by the task flow. After the user provides a 'second user input' (the answer to the query), the device sends this response back to the app to help it finish the task. For example, if you tell your phone, "Order a pizza," the phone identifies a pizza ordering app, gets the steps for ordering from a server, and if the app needs to know the topping, it might display, "What toppings would you like?" on your screen. ## What it does NOT cover - Does not cover scenarios where the 'task flow' (the step-by-step guide) is not received from a 'second electronic device' but is instead entirely pre-programmed or generated on the primary device itself (Claim 1). - Does not cover situations where the digital assistant's 'query' for more information from the user is provided solely through voice, without using a 'touch-sensitive display' (Claim 1). - Does not cover applications that are not 'stored on the electronic device' but run entirely in the cloud or on a remote server (Claim 1). - Does not cover user inputs that are not 'natural-language user input,' such as direct button presses, gestures, or menu selections (Claim 1). - Does not cover systems where the identified software application does not send a 'request' for additional information, meaning the task can be completed without further user interaction (Claim 1). ## The clever bit The clever part is how the system dynamically fetches a 'task flow' from another device to guide interactions with an app, and then uses that flow to generate specific questions for the user on the display when the app needs more details, feeding those answers back to the app. ## Real-world examples 1. Apple Siri interacting with third-party apps like Uber or DoorDash 2. Google Assistant fulfilling requests with installed apps 3. Amazon Alexa routines that involve multiple steps and user confirmation 4. Any modern smartphone digital assistant that can control other applications via voice. ## Why it matters This patent is significant because it outlines a sophisticated way for digital assistants to interact with and control other applications, especially third-party ones. It allows assistants like Siri to handle more complex, multi-step tasks by dynamically asking the user for missing information. This capability is crucial for making digital assistants truly useful beyond simple, built-in commands, expanding their reach into a vast ecosystem of applications. ## Frequently asked questions ### What does How Digital Assistants Control Apps and Ask for More Information cover? This patent describes how a digital assistant on a device can understand what a user wants from a natural language command, find the right app, get a step-by-step guide from another device, and then ask the user for more details on the screen to complete the task with that app. ### Who owns patent US 11204787? Apple Inc owns this patent, granted in 2021. ### When does this patent expire? This patent is expected to expire on December 21, 2041, when the invention enters the public domain. ### What is patent US 11204787 cited by? This patent has been cited by 48 later patents that build on its ideas. ### What problem does this patent solve? This patent is significant because it outlines a sophisticated way for digital assistants to interact with and control other applications, especially third-party ones. It allows assistants like Siri to handle more complex, multi-step tasks by dynamically asking the user for missing information. This capability is crucial for making digital assistants truly useful beyond simple, built-in commands, expanding their reach into a vast ecosystem of applications. ### What does this patent NOT cover? Does not cover scenarios where the 'task flow' (the step-by-step guide) is not received from a 'second electronic device' but is instead entirely pre-programmed or generated on the primary device itself (Claim 1). **Full plain-English explainer:** https://patentbrief.org/patent/us/11204787/github-copilot-code-generation-ai **Original patent:** https://patents.google.com/patent/US11204787 --- _Source: PatentBrief — https://patentbrief.org. Patent facts are from public records; the plain-English explanation is PatentBrief's._