Originality is an important factor in the human perception of value,and search engines factor such human sentiments into their algorithms. Seeing several pages of
duplicated content would not please the user. Accordingly, search engines employ sophisticated algorithms that detect such content and filter it out from search engine results.
Indexing and processing duplicate content also wastes the storage and computation time of a search engine in the first place. if pages are too similar,
then Google [or other search engines] may assume that they offer little value or are of poor content quality.” A web site may not get spidered as often or as comprehensively as a result. And though
it is an issue of contention in the search engine marketing community as to whether there is an explicit penalty applied by the various search engines, everyone agrees that duplicate content can
be harmful.Knowing this, it would be wise to eliminate as much duplicate content as possible from a web site.
It then proposes methods to eliminate or remove it from a search engine’s view. You will:
- Understand the potential negative effects of duplicate content.
- Examine the most common types of duplicate content.
- Learn how to exclude duplicate content using robots.txt and meta tags.
- Use PHP code to properly implement an affiliate program.

Solutions for Commonly Duplicated Pages:
Sometimes the solution is exclusion, other times there are more fundamental solutions addressing web site architecture.
Some of the most frequently observed are the following:
- Print-friendly pages
- Navigation links and breadcrumb navigation
- Affiliate pages
- Pages with similar content
- Pages with duplicate meta tag or title values
- URL canonicalization problems

Print-Friendly Pages
One of the most common sources of duplicate content is the “print-friendly” page. A throwback from the day where CSS did not provide a means to provide multiple media for formatting (print, screen,
and so on), many programmers simply provided two versions for every page — the standard one and the printable one.

Pages with Duplicate Meta Tag or Title Values
A common mistake is to set the meta keywords, meta description, or title values on a web site to the same default value programmatically for every page. If you have complete
duplication of any element (page title, meta keywords, meta description) across your site then it is at best a wasted opportunity, but may also hurt your ability to get your site indexed or ranked well in some search engines.” If
time and resources cannot be dedicated to creating unique meta tags, they should probably not be created at all, because using the same value for every page is certainly not beneficial. Having identical
titles for every page on a web site is also particularly detrimental. Many programmers make this mistake,
because it is very easy not to notice it.

URL Canonicalization
Many web sites exhibit subtle but sometimes insidious duplicate problems due to URL canonicalization problems.

URL-Based Session IDs
URL-based session management causes major problems for search engines, because each time a search engine spiders your web site, it will receive a different session ID and hence a new set of URLs with the
same content. Needless to say, this creates an enormous amount of duplicate content.