Identifying on Page Duplicate Content and how to Avoid Penalties
Duplicate content can potentially destroy websites and get them thrown into the search engine abyss. Maybe that a bit of an exaggeration but duplicate on page content can pose potential harm and risks to your website and cause penalties. Or a reduction in web traffic and lowered search engine visibility. Which is definitely not good for traffic or revenue.
So I am a big fan of free digital marketing tools and I found myself in a situation where I had to crawl through a client’s website to check for some notoriously duplicate content. In order to illustrate how I go about finding duplicate content first make sure you have the free version of screaming frog downloaded and set up on your computer. Screaming frog can be found @
After you have the screaming frog software installed simply copy the url of the website you are trying to crawl and paste it into the bar at the top in within screaming frog. Next, you want to click the meta description tab at the top and hit the duplicate tab.
This will show you which pages on your website have duplicate meta descriptions to other pages. Sometimes this can be an indication of duplicate on page content other times it is simply an indication of just duplicate meta descriptions. Next, you can manually put the url’s into your preferred search engine and check with ones have duplicate on page content.
The next step I like to do is do a quick check for duplicate title tags on a website as well. Simply click page titles and then pan down the drop down and select duplicate. This will give you a list of duplicate page titles as well.
Again, you can manually check each url in your favorite search engine and see which pages have duplicate content on it. If after you have ran your crawl and identified which pages have duplicate or similar content then you can begin devising a strategy on how to combat this issue.
Moving forward a good strategy may be to combine your duplicate content to form a stronger page that may rank better in search engines. For example, if you have an e-commerce site and have similar products with similar on page content you could 301 to whichever page has a better page rank. Moz has a great tool that you can download for free as an extension and comb through your pages to determine which ones rank better in this instance. Then from there you can devise an appropriate strategy. Or on the other hand if you notice you have similar on page content then you can move forward and further build out the pages to make them different and avoid incurring any penalties.
I hope this helps with identifying on page duplicate content. Please share your duplicate content stories and how you deal with them.