Duplicate content checker

It normalizes case and punctuation, builds a set of phrases from each text, and measures their overlap with the Jaccard coefficient. You can choose the shingle size and the percentage that should trigger a likely-duplicate flag. The result is deterministic, easy to audit, and useful for reviewing landing pages, product descriptions, syndicated articles, and other copy before publication.

What shingle similarity reveals

A shingle is a short run of consecutive words. With a shingle size of three, the sentence “write useful pages for readers” produces phrases such as “write useful pages,” “useful pages for,” and “pages for readers.” The checker creates these phrase sets for both texts, counts the phrases they share, and divides that count by the number of unique phrases across both sets. That set-based Jaccard calculation produces the similarity percentage. It rewards matching passages while ignoring capitalization and punctuation differences that do not change the words. This makes the result more informative than comparing individual word counts, because common vocabulary alone does not create a match unless the words occur in the same local order. The score is evidence of textual overlap, not a search-engine penalty prediction. A high value tells you that the submitted versions reuse many of the same phrases and deserve editorial review. A low value tells you their exact phrasing differs, although it cannot prove that either page is original, accurate, useful, or safe to publish.

Choose a shingle size and threshold

The default shingle size is three words, a practical starting point for ordinary web copy. Smaller shingles are more sensitive: they catch short shared fragments, but they can also count routine phrases that appear naturally across unrelated pages. Larger shingles demand longer verbatim sequences, so they reduce incidental matches while overlooking passages that were lightly rewritten. Choose the size before interpreting the percentage, because scores produced with different sizes are not directly comparable. The duplicate threshold controls only the returned likely-duplicate flag; it does not alter the underlying similarity calculation. A threshold of 70 means a score of 70 percent or higher is flagged. Teams should calibrate that value with examples from their own content library instead of treating one universal cutoff as an SEO rule. Keep the same settings when comparing batches so results remain consistent. Each submitted text must contain at least as many normalized words as the shingle size. If either one is shorter, the request fails instead of presenting an artificial or mathematically misleading percentage.

Turn the result into an editorial decision

Use the score as a screening signal in a review workflow. Start by comparing pages that target similar queries, templates that were populated from the same catalog, or a draft against the source material supplied to a writer. When the tool flags a pair, inspect both documents and identify why their shingles overlap. Boilerplate navigation, legal notices, specifications, and standard calls to action may be legitimate. Matching introductions, explanations, benefits, and conclusions are stronger reasons to revise. Preserve facts that must remain correct, but rewrite the organization and supporting language so each page serves its own reader intent. Record the shingle size and threshold beside the result if you need a reproducible audit trail. The checker processes exactly two texts per request and does not crawl URLs, search the public web, identify an original author, or decide which page a search engine will rank. For a site-wide review, select meaningful candidate pairs first, run them with consistent settings, and prioritize the highest scores for human examination rather than automatically deleting or redirecting pages.