<http://schema.org/articleBody>
"Duplicate Content and Multiple Site IssuesModerated by Anne Kennedy from Beyond Ink, who also has a short presentation.\n\u201CDouble Trouble\u201D What is duplicate content?  Multiple domains: identical homepage, different URL\u2019s.  Different links to several URL\u2019s from one site.  The reasons not to do this: the SE\u2019s say so.  That is enough.  Confusing the robot (crawler): Mirror sites (one website multiple domains).  Uses an example of the site for the International Research Foundation for RDS/CRPS.  They decided instead of using the (.org) Top Level Domain, they thought it would be better to be a (.edu) TDL.  When they did this, it disappeared from view.  This is a note to watch for IT types that think they may know how to optimize for SE\u2019s.  Another example: Lifeline Systems had 2 URL\u2019s.  Previously, all business came from health care professionals.  So many links came from various health care partners and hospitals.  When they decided to launch a consumer campaign, they changed URL and caused much confusion.  They ended up getting banned, but it was fixed.Dynamic URL\u2019s: short introduction because Mikkel will be addressing, but look for pages in results without real Tile or description and the \u201Cwe have removed pages that were similar\u201D at the bottom of the results.  301 redirects are the \u201Chero.\u201D  Highly recommends using these in the case of removing duplicate content.  If you have been banned, use Google\u2019s and Yahoo\u2019s reinclusion requests.Shari Thurow from Grandtastic Designs\nWill talk about some elements to the duplicate content filter, and why SE\u2019s like to filter dupe content.  Understand that duplicate content is an unclear definition.  It can be an exact copy, or simply the change of a date or something that is \u201Cnear-duplicate\u201D content.  #1 reason is that too much dupe content interferes with the retrieval of information process.  SE\u2019s and consumers both want fast results.  They learned how to cluster by limiting results from one particular site to two maximum from a website.  They omit duplicate results with the note that Anne mentioned above.When SE\u2019s look to determine uniqueness of each document, they use a \u201Cboiler plate\u201D approach.  This is also known as \u201Cblock level analysis.\u201D  They omit the navigation bars, and focus on the main content areas.  They look for every single webpage to have unique in-links (linkage properties).  If two URL\u2019s have the same in-links, there is a possibility of duplicate content.  In the case of articles, they can be represented in more than one site because of the unique boiler plate and the unique linkage properties.  They also look for \u201Ccontent evolution.\u201D \u201CIn general 65% of web content does not change on a weekly basis. 0.8% will change completely on a weekly basis, such as a news site.  They are looking at the \u201Caverage page mutation\u201D of a URL and a website.  If too high, and the boiler plates are too similar, they will filter out the duplicate results.Another type of filter is the \u201Chost name resolution.\u201D  Uses BMW example.  If the host name resolves to the same company or organization, that organization is probably monopolizing control of content.  If content is moved to different servers too often, they will look for that too, since \u201Csearch engine spammers\u201D tend to do this.  Last thing: she is an Andre Broder \u201Cgroupie.\u201D All SE\u2019s uses his concept of \u201Cshingle comparison.\u201D  You can search for a detailed explanation, but essentially, the more shingles of word groups (word sets) that are available on multiple URLs, the higher the likelihood of dupe content.  SE\u2019s only want one page.  Shari doesn\u2019t like when SE\u2019s pick which one to display.  She suggests determining the URL\u2019s you want displayed if you have similar descriptions, and robots.txt exclude the other ones.Some duplicate content is considered spam.  This is where clustering has come in handy.  The ability for SE\u2019s to determine dupe content has improved greatly even since NYC SES.  Shows an example of a University website that has a \u201Challway\u201D page which is a \u201Csitemap\u201D of their doorway pages.  The example has a link to \u201Cmarketing degree,\u201D followed by a link to \u201Cmarketing degreeS.\u201D  Some duplicate content is a copyright infringement.  She recommends using CopyScpae.com to help find people \u201Cswiping your content.\u201D  Also suggests using the \u201CWaybackmachine\u201D at archive.org to find copies of old content that may be in there and have been copied.  Keep your own records with versions of your content.  You have to prove this to the SE\u2019s because they cannot take it down just on your word.Summary: use Robots exclusion, have a copyrighting plan, keep track of you content, register with copyright.gov.  Last but not least: don\u2019t exploit the SE\u2019s and help everyone\u2019s experience.Mikkel deMib Svendsen from RedZoneGlobal\nWill be covering some of the technical duplicate issues.  The list is pretty long, so he could not possibly go through the entire list, so he will focus on more common tech issues: With and without \u201CWWW\u201D (canonical issues).  Session ID\u2019s, URL rewriting, many-to-one problems in forums, sort order parameters and breadcrumb navigation.WWW or not WWW?  Indexing issues: most engines seem to be able to deal with this.  Linking issues: yes, still an issue, because you want people to link to the URL\u2019s that the engines are prioritizing.  So he recommends using 301 redirect to the most popular one.  The WWW version seems to be most common.Session ID\u2019s can be a true nightmare.  A website (smartpages.com) had 200,000 versions of its homepage indexed.  Recommended solution is to dump session info into a cookie for all users, or id spiders and feed them their own content.  Google recommends this \u2013 it is not \u201Ccloaking.\u201D  You are striping out info to help the engines.Wordpress and other blog solutions.  Rewrite URL\u2019s without parameters by identifying what you want used (such as post name and /or post ID).  The problem is that when you do this, you still have the re-written URL and the original URL.  Even thought here are no links to this, the engines end up fin ding them.  It only takes one link to the original version for it to be crawled and a dupe issue.  Once again, use 301 redirects to fix.  In Wordpress there is a plugin you can use to automatically 301 redirect the original to the next.  Uses a tool (Schlueterica Wordpress Canonical URL plugin) for this (isaacschlueter.com/plugins/i-made/cannonical-url/).\u201CMany-to-one\u201D in forums where a page can be requested in different manners, such as the URL or the URL appended with a Post ID and/or a Thread Page ID.  There are some workarounds, he feels the one SEW is using could be improved.  There is no linking solution.  His idea is once again that you have to 301 redirect.  Sort order parameters in the URL will cause duplicate indexing as well. Is yet another waste of the link popularity.Breadcrumb navigation: you may have a problem if the navigation, you may have  a problem, especially if you categorize a product in multiple ways.  The first thing is that unique pages should also have one unique URL.  If you want to reflect breadcrumbs, he suggests storing them in a cookie.  Remember that we have infinite ways to create multiple URL\u2019s to a different page.QA Speakers include Tim Converse from Yahoo!, Matt Cutts from Google, and Rahul Lahiri from Ask.comTim: all of the info that was presented was very accurate.  He wants to distinguish inadvertent duplication from \u201Cspamming.\u201D  The perfect thing for a crawler would be to have one URL for every unique page, but he knows that people don\u2019t necessarily think about SE\u2019s only when designing.  Do not worry about getting banned for inadvertent issues.  The easier you can make it for us, the better it will be.  The ultimate goal for everyone is to present a diverse result for the user.  If you stray into creating multiple domains with the same content or \u201Crepurposing\u201D someone else\u2019s content, they find this to be abusive.  Ask yourself \u201Cdo I own the content?\u201DMatt: again very good information.  Of course the goal is to have good content.  There is definitely abusive behavior.  People often ask,\u201D I have a .DE and a .FR of a site.\u201D  This is not a problem.  If you are taking the time to translate in to multiple languages, no problem.  If you have an article in four pages also rendered in one page, that is ok, but suggest blocking the spider from one.  If you have \u201Cnear duplicates,\u201D with different boiler plates but identical/similar main content.  When G crawls and presents search results, even to the last millisecond, they remove possible duplicates.  Last point: link consistency.  He would recommend that all links be to WWW.  Just this past Friday, they renamed Sitemaps to Google Webmaster Tools.  There is a tool there that if you want to notify them that you \u201Cown\u201D the link at DMOZ that goes to the non-www version, you can report it and they will take it into consideration.Rahul also aggress with the other speakers and discusses a couple examples of duplicate content and how they are trying to deal with it.  He discusses how the Nestle Brands of water which are branded differently in different geographical areas (Poland Springs, etc\u2026) each have their own URL, with identical sites that only change the brand name."^^<http://www.w3.org/2001/XMLSchema#string>
<https://www.seroundtable.com/archives/004334.html>