Answers.Body: <p>First you would need to figure out how to identify duplicates.</p>
  
  <p>Craft doesn’t allow duplicate entry slugs; it will append “-1”, “-2”, etc. to them to keep them unique. So that would probably be a good starting point.</p>
  
  <p>Use this query to get a list of all entries that have a number appended to the end of the slug. It’ll tell you the entry title, its section’s name, and the URL to edit it in the control panel. (Just update “example.com” with the actual host name.) Go through those and use your best judgement to decide which ones should be deleted.</p>
  
  <pre><code>SELECT c.title, s.name AS sectionName, CONCAT('http://example.com/admin/entries/', s.handle, '/', el.elementId) AS cpUrl
  FROM craft_elements_i18n el
  INNER JOIN craft_content c ON c.elementId = el.elementId
  INNER JOIN craft_entries en ON en.id = el.elementId
  INNER JOIN craft_sections s ON s.id = en.sectionId
  WHERE el.slug REGEXP '\-[0-9]+$'
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-06-30T18:38:44.207Z
Answers.Text: Hi Brandon, thanks - that's a really useful piece of sql magic! However, I got two problems here: because my entries are imported with Bob Olde Hampsink's Import plugin I actually seem to have ended up with duplicate slugs. Secondly many of my entries have titles like this "Coop Obs Jessheim avd. 223000" with numbers in them. In lack of a better solution I'll try to manually remove the duplicates in the csv-files.
  
  Thanks for a very useful code-snippet anyway!
Answers.UserDisplayName: alexbech
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-06-30T18:57:44.667Z
Answers.Text: The import plugin would have the same restrictions. Technically, slugs can be the same as long as the resulting URR would be different (e.g. the same slug across 2 different sections, or 2 different locales).
Answers.UserDisplayName: Brandon Kelly
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-06-30T19:05:50.107Z
Answers.Text: Ah I see, I've unchecked the "Entries in this section have their own URLs" - could that be it? Anyway, I modified your sql to just list duplicates based on title.
  
  `SELECT c.title, count(c.title) as times, s.name AS sectionName
  FROM craft_elements_i18n el
  INNER JOIN craft_content c ON c.elementId = el.elementId
  INNER JOIN craft_entries en ON en.id = el.elementId
  INNER JOIN craft_sections s ON s.id = en.sectionId
  GROUP BY c.title
  HAVING times>1`
  
  It turns out there are quite a few duplicates, now to figure out a safe way to delete them.
Answers.UserDisplayName: alexbech
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-06-30T19:17:46.157Z
Answers.Text: Ah yes -- the "-1"s only get appended when Craft is trying to come up with a unique URI. Good call.
Answers.UserDisplayName: Brandon Kelly
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-06-30T10:21:35.757Z
Answers.LastActivityDate: 2014-06-30T10:21:35.757Z
Answers.UserDisplayName: Brandon Kelly
Body: <p>I have imported some data for a client, and it turns out there are some duplicates in the csv-files.</p>
  
  <p>Is there an easy way to remove or filter out duplicates, either in my entries-query or in my template?</p>
  
  <p>My query looks like this now</p>
  
  <pre><code>{% set dealers = craft.entries({section:'dealers'}).order('dealerName') %}
  </code></pre>
  
  <p>Thanks!</p>
  
Tags: <entries><templating>
Title: Is there a way to remove or filter out duplicate entries?
ThreadText: # Is there a way to remove or filter out duplicate entries?
  
  (Asked by: username_0 on 2014-06-30)
  
  I have imported some data for a client, and it turns out there are some duplicates in the csv-files.
  
  Is there an easy way to remove or filter out duplicates, either in my entries-query or in my template?
  
  My query looks like this now
  
  ```
  {% set dealers = craft.entries({section:'dealers'}).order('dealerName') %}
  
  ```
  
  Thanks!
  
  **Tags:** entries, templating
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2014-06-30. Score: 3 | Accepted answer
  
  First you would need to figure out how to identify duplicates.
  
  Craft doesn’t allow duplicate entry slugs; it will append “-1”, “-2”, etc. to them to keep them unique. So that would probably be a good starting point.
  
  Use this query to get a list of all entries that have a number appended to the end of the slug. It’ll tell you the entry title, its section’s name, and the URL to edit it in the control panel. (Just update “example.com” with the actual host name.) Go through those and use your best judgement to decide which ones should be deleted.
  
  ```
  SELECT c.title, s.name AS sectionName, CONCAT('http://example.com/admin/entries/', s.handle, '/', el.elementId) AS cpUrl
  FROM craft_elements_i18n el
  INNER JOIN craft_content c ON c.elementId = el.elementId
  INNER JOIN craft_entries en ON en.id = el.elementId
  INNER JOIN craft_sections s ON s.id = en.sectionId
  WHERE el.slug REGEXP '\-[0-9]+$'
  
  ```
  
  > **Comments:**
  > **username_0**: Hi username_1, thanks - that's a really useful piece of sql magic! However, I got two problems here: because my entries are imported with Bob Olde Hampsink's Import plugin I actually seem to have ended up with duplicate slugs. Secondly many of my entries have titles like this "Coop Obs Jessheim avd. 223000" with numbers in them. In lack of a better solution I'll try to manually remove the duplicates in the csv-files.  Thanks for a very useful code-snippet anyway!  
  > **username_1**: The import plugin would have the same restrictions. Technically, slugs can be the same as long as the resulting URR would be different (e.g. the same slug across 2 different sections, or 2 different locales).  
  > **username_0**: Ah I see, I've unchecked the "Entries in this section have their own URLs" - could that be it? Anyway, I modified your sql to just list duplicates based on title.  `SELECT c.title, count(c.title) as times, s.name AS sectionName FROM craft_elements_i18n el INNER JOIN craft_content c ON c.elementId = el.elementId INNER JOIN craft_entries en ON en.id = el.elementId INNER JOIN craft_sections s ON s.id = en.sectionId GROUP BY c.title HAVING times>1`  It turns out there are quite a few duplicates, now to figure out a safe way to delete them.  
  > **username_1**: Ah yes -- the "-1"s only get appended when Craft is trying to come up with a unique URI. Good call.  
  
  ---
  
  