How AI Uses Question-Guided Attention to Answer Questions About Images
A method for AI to answer questions about images by dynamically focusing on relevant parts of the picture based on the specific question asked.
Original patent title: “Systems and methods for attention-based configurable convolutional neural networks (ABC-CNN) for visual question answering”
A method for AI to answer questions about images by dynamically focusing on relevant parts of the picture based on the specific question asked. Granted to Baidu USA LLC in 2018 with 23 claims and 28 forward citations, and it is expected to expire in 2036.
Coverage
What does this patent actually cover?
This patent describes a way to make AI better at answering questions about photos or videos. It uses a system called ABC-CNN that takes both an image and a text question as input. The system extracts features from the image and turns the question into a mathematical representation. It then uses the question to create special 'kernels'—small filters that act like a spotlight—to highlight only the parts of the image relevant to the question. Finally, it combines this focused information with the original image data to generate an accurate answer, effectively ignoring irrelevant background noise.
The gap
What does this patent NOT cover?
- Does not cover general-purpose image classification that does not involve a natural language question input.
- Does not cover attention mechanisms that are not specifically implemented via configurable convolutional kernels.
- Does not cover non-neural network methods for image analysis or traditional rule-based computer vision.
- Does not cover the specific hardware used to run the neural network, only the algorithmic method.
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
Key facts
What made this novel
The innovation is using the question itself to dynamically generate the convolutional kernels, effectively letting the text input 'program' the AI's visual focus mechanism on the fly.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
Visual Question Answering (VQA) systems
Smart camera search features
Automated accessibility tools for the visually impaired
Advanced content moderation systems
Why it matters
The bigger picture
This technology is a building block for multimodal AI, which allows computers to 'see' and 'understand' the world in a human-like way. By enabling AI to selectively focus on specific regions of an image, it significantly improves the accuracy of tasks like automated image captioning and visual search. It represents a shift from static image processing to dynamic, context-aware analysis.
Filed
June 16, 2016
Granted
May 8, 2018
Market context
Who's building on this
Companies in this space
Baidu continues to be a major player in this space, integrating these techniques into their search and autonomous driving platforms. Other major tech companies like Google, Meta, and Microsoft are also heavily invested in similar attention-based multimodal architectures for their respective AI assistants and vision models.
Market impact
This patent contributed to the maturation of visual-language models, moving the industry toward more efficient and accurate multimodal AI. It helped standardize the use of attention mechanisms in VQA tasks, which is now a foundational capability for modern large multimodal models like GPT-4o or Gemini.
Claim 1 — Plain English
What this patent covers
This patent describes a way to make AI better at answering questions about photos or videos. It uses a system called ABC-CNN that takes both an image and a text question as input. The system extracts features from the image and turns the question into a mathematical representation. It then uses the question to create special 'kernels'—small filters that act like a spotlight—to highlight only the parts of the image relevant to the question. Finally, it combines this focused information with the original image data to generate an accurate answer, effectively ignoring irrelevant background noise.
The clever bit
The innovation is using the question itself to dynamically generate the convolutional kernels, effectively letting the text input 'program' the AI's visual focus mechanism on the fly.
What it does not cover
- Does not cover general-purpose image classification that does not involve a natural language question input.
- Does not cover attention mechanisms that are not specifically implemented via configurable convolutional kernels.
- Does not cover non-neural network methods for image analysis or traditional rule-based computer vision.
- Does not cover the specific hardware used to run the neural network, only the algorithmic method.
Patent timeline
Application submitted to the patent office
Application published, typically 18 months after filing
Patent officially issued
Patent enters public domain
PatentBrief Score
Impact Score
Moderate
Citation count
29/40
Moderately cited
Claim breadth
15/20
Broad claimsclaimsThe numbered statements at the end of a patent that legally define what the inventor owns.Read more →
Recency
10/20
Granted 5–10 years ago
Assignee scale
0/20
Independent or smaller assigneeassigneeThe entity that owns the patent — usually the inventor's employer or a company.Read more →
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$156K – $499K
Midpoint $312K · 9.9 yr remaining · industry ×1.6
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
Claim text not yet imported for this patent
The original legal language
Original claims
23 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
Chen, K., Xu, W., & Wang, J. (2018). How AI Uses Question-Guided Attention to Answer Questions About Images (U.S. Patent No. 9,965,705). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/9965705/systems-and-methods-for-attention-based-configurable-convolutional-neural-networks-abc-cnn-for-visual-question-answering
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US9965705"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4965188 · 1990
How to Make Many Copies of a DNA Piece with Heat
This patent describes the Polymerase Chain Reaction (PCR) method, a technique to make millions of copies of a specific DNA segment using a heat-resistant enzyme and repeated temperature changes.
Cetus Corp
US 4235871 · 1980
How to Encapsulate Active Materials in Lipid Bubbles Efficiently
This patent describes a method for trapping biologically active substances inside tiny, multi-layered fat bubbles called liposomes, using a specific water-in-oil emulsion and gel-forming process to improve how much material gets captured.
Individual
Semantically similar
You might also find these interesting
US 10452978 · 2019 · Google LLC
How AI Models Understand Language Using 'Attention'
US 20230162023 · Mitsubishi Electric Research Laboratories
Automated AI for Adapting to New Data Without Retraining
US 11023780 · 2021 · Superb AI Co Ltd
How AI Automatically Labels Images and Checks Its Own Work
US 20250363357 · Ubotica Technologies
How to Update AI on Small Devices with Slow Internet
More to explore
More in AI & Machine Learning
US 10452978 · 2019 · Google LLC
How AI Models Understand Language Using 'Attention'
US 6523026 · 2003 · Huntsman International LLC
How Computers Find Hidden Connections Between Different Fields of Knowledge
US 11615208 · 2023 · Capital One Services LLC
How Cloud Systems Automatically Create and Train AI Data Models
US 10402750 · 2019 · Facebook Inc
How Facebook Uses Deep Learning to Predict What You Might Like
New to patents?
Common Questions
Frequently Asked Questions
What does How AI Uses Question-Guided Attention to Answer Questions About Images cover?
A method for AI to answer questions about images by dynamically focusing on relevant parts of the picture based on the specific question asked.
Who owns patent US 9965705?
Baidu USA LLC owns this patent, granted in 2018.
When does this patent expire?
This patent is expected to expire on June 16, 2036, when the invention enters the public domain.
What is patent US 9965705 cited by?
This patent has been cited by 28 later patents that build on its ideas.
What problem does this patent solve?
This technology is a building block for multimodal AI, which allows computers to 'see' and 'understand' the world in a human-like way. By enabling AI to selectively focus on specific regions of an image, it significantly improves the accuracy of tasks like automated image captioning and visual search. It represents a shift from static image processing to dynamic, context-aware analysis.
What does this patent NOT cover?
Does not cover general-purpose image classification that does not involve a natural language question input.
Patent monitoring