How Google Uses User Feedback to Rank Websites by Quality
A method for Google to judge a website's quality by looking at how users interact with its search results across different categories of search queries.
Original patent title: “Evaluating website properties by partitioning user feedback”
A method for Google to judge a website's quality by looking at how users interact with its search results across different categories of search queries. Granted to Google LLC in 2013 with 36 claims and 26 forward citations, and it is expected to expire in 2030.
Coverage
What does this patent actually cover?
This patent describes a way to measure the quality of a website by grouping search data into specific buckets. It takes pairs of documents and the queries that led people to them, then sorts these pairs into partitions based on how well the document matched the query. By looking at how users click on results within these different buckets, the system calculates a 'skew' or statistical distribution of feedback. If a website performs unexpectedly across these partitions, it indicates a specific level of quality, which is then used as a signal to rank that website higher or lower in future search results.
The gap
What does this patent NOT cover?
- Does not cover ranking websites based on simple, non-partitioned metrics like total click count alone.
- Does not cover manual human review of websites for quality assessment.
- Does not cover ranking methods that ignore the relationship between the query and the specific document retrieved.
- Does not cover systems that do not use statistical skew as a primary indicator of quality.
These exclusions are unique to PatentBrief — derived from the actual claim language, not patent-office boilerplate.
Key facts
What made this novel
The innovation is using 'skew'—a measure of how lopsided a data distribution is—to detect quality. By comparing how a site performs on high-relevance vs. low-relevance queries, the system can identify if a site is genuinely useful or just gaming the system.
The Patent Drawing
Schematic visualization of the patent's claim structure. Hand-drawn diagrams in progress for each landmark patent.
Where you've seen this
Real-world examples
Google Search ranking algorithms
Automated website quality scoring systems
Search engine result page (SERP) relevance tuning
Why it matters
The bigger picture
This patent is part of the technical foundation for modern search engine optimization (SEO) and ranking algorithms. It helps search engines distinguish between high-quality content and 'spammy' sites that might get clicks but don't actually satisfy user intent. It is a core component of how Google maintains the relevance of its search results at scale.
Filed
February 3, 2010
Granted
December 24, 2013
Market context
Who's building on this
Companies in this space
Google continues to refine these techniques as part of its core search infrastructure. Other major search providers and large-scale web crawlers utilize similar statistical partitioning methods to manage the massive influx of user interaction data.
Market impact
This technology enabled search engines to move beyond simple keyword matching toward intent-based ranking. It forced website owners to focus on providing genuine value to users rather than just optimizing for search engine crawlers, fundamentally shaping the modern digital marketing landscape.
Claim 1 — Plain English
What this patent covers
This patent describes a way to measure the quality of a website by grouping search data into specific buckets. It takes pairs of documents and the queries that led people to them, then sorts these pairs into partitions based on how well the document matched the query. By looking at how users click on results within these different buckets, the system calculates a 'skew' or statistical distribution of feedback. If a website performs unexpectedly across these partitions, it indicates a specific level of quality, which is then used as a signal to rank that website higher or lower in future search results.
The clever bit
The innovation is using 'skew'—a measure of how lopsided a data distribution is—to detect quality. By comparing how a site performs on high-relevance vs. low-relevance queries, the system can identify if a site is genuinely useful or just gaming the system.
What it does not cover
- Does not cover ranking websites based on simple, non-partitioned metrics like total click count alone.
- Does not cover manual human review of websites for quality assessment.
- Does not cover ranking methods that ignore the relationship between the query and the specific document retrieved.
- Does not cover systems that do not use statistical skew as a primary indicator of quality.
Patent timeline
Application submitted to the patent office
Application published, typically 18 months after filing
Patent officially issued
Patent enters public domain
PatentBrief Score
Impact Score
Strong
Citation count
29/40
Moderately cited
Claim breadth
20/20
Very broad protection
Recency
5/20
Granted 10–20 years ago
Assignee scale
20/20
Major company or institution
PatentBrief Impact Score — based on citation count, claim breadth, recency, and assignee scale. Not a legal assessment.
Heuristic Value Estimate
What this patent might be worth
$202K – $645K
Midpoint $403K · 3.6 yr remaining · industry ×1.6
Heuristic only — blends forward/backward citation counts, claim scope, time remaining, litigation history, and CPC-derived industry baseline. Real valuations need a professional appraisal.
Claim text not yet imported for this patent
The original legal language
Original claims
36 claims as filed with the patent office.
Concepts involved
Citations
Patent lineage
Cite this patent
Flaster, M., Fernandes, N. C., & Lehman, A. R. (2013). How Google Uses User Feedback to Rank Websites by Quality (U.S. Patent No. 8,615,514). U.S. Patent and Trademark Office. https://patentbrief.org/patent/us/8615514/evaluating-website-properties-by-partitioning-user-feedback
Auto-generated from the patent record. Double-check author order and the issue date against the official USPTO document before submitting.
Embed
Add this patent to your site
Drop this plain-English patent card into any blog post or article — free, no signup. It always links back to the full breakdown here.
<div data-patentlens-widget data-patent-number="US8615514"></div> <script src="https://patentbrief.org/embed.js" async></script>
Stay in the loop
Get a weekly digest of new patents.
One email per week. No spam. Unsubscribe anytime.
Keep exploring
Related patents you should know
US 4683195 · 1987
How to Make Billions of Copies of a DNA Segment
This patent describes the Polymerase Chain Reaction (PCR), a method to rapidly create many copies of a specific piece of DNA or RNA, enabling its detection and analysis.
Cetus Corp
US 8697359 · 2014
How to Edit Genes in Human Cells Using an Engineered CRISPR System
This patent describes an engineered CRISPR-Cas9 system for precisely cutting DNA in eukaryotic cells to change how genes work, opening the door for gene editing in complex organisms.
Massachusetts Institute of Technology
US 7657849 · 2010
How the iPhone's Slide-to-Unlock Gesture Works
Apple's 2010 patent describes unlocking a device by dragging a specific graphical image across the touchscreen along a predefined path, a gesture that became iconic with the original iPhone.
Apple Inc
US 4733665 · 1988
How Doctors Implant a Permanent Stent Using a Balloon
This patent describes the method for placing a permanent, expandable wire mesh tube inside a blood vessel or other body tube using a balloon-tipped catheter to widen it and keep it open.
Expandable Grafts Partnership
US 4965188 · 1990
How to Make Many Copies of a DNA Piece with Heat
This patent describes the Polymerase Chain Reaction (PCR) method, a technique to make millions of copies of a specific DNA segment using a heat-resistant enzyme and repeated temperature changes.
Cetus Corp
US 4235871 · 1980
How to Encapsulate Active Materials in Lipid Bubbles Efficiently
This patent describes a method for trapping biologically active substances inside tiny, multi-layered fat bubbles called liposomes, using a specific water-in-oil emulsion and gel-forming process to improve how much material gets captured.
Individual
Semantically similar
You might also find these interesting
US 6285999 · 2001 · Leland Stanford Junior University
How Websites Get Ranked by Importance
US 6370526 · 2002 · International Business Machines Corp
Smart Ranking of Emails and Files Based on How You Click
US 8914392 · 2014 · Facebook Inc
How Facebook Ranks Search Results Based on Your Friends' Activity
US 9268820 · 2016 · Google LLC
How Google Displays Knowledge Panels Next to Search Results
More to explore
More in Software & Internet
US 4405829 · 1983 · Massachusetts Institute of Technology
How RSA Public-Key Encryption Keeps Digital Messages Secret
US 6285999 · 2001 · Leland Stanford Junior University
How Websites Get Ranked by Importance
US 5960411 · 1999 · Amazon com Inc
How Amazon's One-Click Ordering Works for Online Purchases
US 7669123 · 2010 · Facebook Inc
Displaying Friends' Activities in a Social Network Feed
New to patents?
Common Questions
Frequently Asked Questions
What does How Google Uses User Feedback to Rank Websites by Quality cover?
A method for Google to judge a website's quality by looking at how users interact with its search results across different categories of search queries.
Who owns patent US 8615514?
Google LLC owns this patent, granted in 2013.
When does this patent expire?
This patent is expected to expire on February 3, 2030, when the invention enters the public domain.
What is patent US 8615514 cited by?
This patent has been cited by 26 later patents that build on its ideas.
What problem does this patent solve?
This patent is part of the technical foundation for modern search engine optimization (SEO) and ranking algorithms. It helps search engines distinguish between high-quality content and 'spammy' sites that might get clicks but don't actually satisfy user intent. It is a core component of how Google maintains the relevance of its search results at scale.
What does this patent NOT cover?
Does not cover ranking websites based on simple, non-partitioned metrics like total click count alone.
Same assignee
More from Google LLC
Patent monitoring