Duplicate Content: What Actually Penalises You

Almost every week someone asks whether their site has been hit by the duplicate content penalty. In the sense they mean it — a punishment applied because two pages resemble each other — that penalty does not exist. Google has said so publicly since 2008, and the current documentation says the same thing in plainer language.

That does not mean repetition is free. It means the cost is different from the one people fear, and the fix is different too.

What Google actually does with repeated pages

When Google finds several URLs serving substantially the same content, it groups them into a cluster and picks one to represent the group in search results. That chosen URL is the canonical. The others are still crawled and still known, but they are filtered out of the results so users do not see the same page five times.

Your rel="canonical" tag participates in that decision. It does not decide it. Google treats a canonical declaration as a strong hint alongside other signals — internal links, redirects, the sitemap, which version has the inbound links, which one is on HTTPS. If those signals disagree with your tag, Google can and does choose differently.

You can see the outcome directly. In Search Console, open the URL Inspection tool for any page and compare the “user-declared canonical” line with the “Google-selected canonical” line. Where they differ, Google has overruled you, and the Pages report will list those URLs under “Duplicate, Google chose different canonical than user”.

The real cost of duplication

None of this is a punishment. It is still a problem, for three reasons.

Signals get split. If half your inbound links point at one version of a page and half at another, the consolidated authority that should sit on one URL is spread across two. Canonical tags and redirects exist mainly to merge those signals back together.

The wrong URL wins. Google picks a representative, and it will not always pick the one you optimised, the one with the tracking parameters stripped, or the one your campaign links to. Ranking with an ugly parameter URL is a common and entirely avoidable symptom.

Crawling is wasted. On a small brochure site this is irrelevant. On a large catalogue with faceted filters generating tens of thousands of near-identical URLs, crawlers spend their time on combinations nobody searches for while new pages wait.

The duplicates that matter more than your copy

Most duplication is technical, not editorial. These are the ones to check first, because they are invisible until you look and cheap to fix once you do.

• Protocol and host variants. http:// and https://, www and non-www. All four should resolve to one, by redirect, not by canonical tag alone.
• Trailing slash inconsistency. /services and /services/ are different URLs to a crawler.
• Tracking and session parameters. Campaign tags, sort orders and filters appended to a URL create a new address for the same page every time.
• Index pagination. Page two of a listing that serves identical intro copy and a different batch of links.
• Printer-friendly and AMP-style alternates left behind by an old theme.
• A staging or development site that got indexed. This one is worth checking today. Search site: your staging domain. If it returns results, the staging build has been competing with the live one, and neither password nor noindex was in place.

Fixing these is usually a server or CMS configuration change rather than a writing job. It belongs with whoever handles your website build.

What genuinely gets a site in trouble

The documentation is specific here. Duplicate content becomes grounds for action when it appears that the intent is to be deceptive and to manipulate results. That is a narrow category, and it is about intent and scale rather than similarity.

In practice, the patterns that attract trouble look like this:

Scaled pages generated from a template. Fifty location pages where the only change between them is the town name. This is the single most common self-inflicted wound in local SEO. Each page targets a place, none of them says anything true about that place, and collectively they read as what they are.

Content lifted from elsewhere and republished with no added value — scraped manufacturer descriptions, syndicated articles, spun rewrites of a competitor’s guide.

Doorway pages built to catch a search and funnel everyone to the same destination.

The distinction that matters: repeating yourself because of a URL quirk is a housekeeping issue. Mass-producing near-identical pages to occupy more of the results page is a different thing entirely, and no canonical tag makes it acceptable.

The things people worry about that are fine

Your header, footer and navigation appearing on every page. Boilerplate is expected. Search engines have handled it for two decades.

Your address and opening hours repeated site-wide. That is consistency, and it helps.

A product available in six sizes with six URLs. Normal for ecommerce. Pick a canonical, or use a single page with variants.

Quoting a source with attribution. Quoting is not duplication.

Your own press release picked up by three trade publications. Cross-domain duplication of this kind is legitimate. Google will choose one to show for a given query; that is not a penalty against you.

Manufacturer descriptions on an ecommerce site. This one is a genuine competitive weakness rather than a risk. If your product page says exactly what forty other retailers say, there is no reason for Google to prefer yours. The answer is to add something — sizing notes, use cases, photographs of the actual item — not to fear a sanction.

How to audit this in an afternoon

1. Crawl your own site with any desktop crawler and sort by title tag. Repeated titles are the fastest way to spot repeated pages.
2. In Search Console, open the Pages report and read the “Why pages aren’t indexed” section. The duplicate categories there are Google telling you exactly what it clustered.
3. Inspect a handful of important URLs and check the declared canonical against the selected one.
4. Search site:yourdomain.com with a distinctive sentence from a key page in quotes. If more than one of your own URLs comes back, you have found a cluster.
5. Check that your redirects actually fire. Type the http, www and trailing-slash variants of your homepage into a browser and watch where they land.

The honest summary

Duplication is a plumbing problem dressed up as a content problem. Fix the URL variants, declare your canonicals, and stop worrying about whether two service pages share a paragraph of boilerplate.

The thing worth worrying about is the opposite failure: pages that exist only to occupy a keyword and give a reader nothing. That is not penalised for being duplicate. It is ignored for being useless, which costs you the same either way.

If you are not sure which category your site falls into, a technical review will tell you in a morning.