Answers.Body: <p>Unfortunately, the dedupe screen provided by the CiviCRM searches contacts outside the group.</p>
  <p>But I have found <a href="https://civicrm.org/extensions/deduper" rel="nofollow noreferrer">Deduper extension</a> much useful for larger databases.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-28T14:57:34.320Z
Answers.LastActivityDate: 2024-05-28T14:57:34.320Z
Answers.UserDisplayName: Pradeep Nayak
Answers.Body: <p>You could also have a look at our extension <a href="https://github.com/fuzionnz/nz.co.fuzion.dedupegroup" rel="nofollow noreferrer">here</a></p>
  <blockquote>
  <p>Adds the potential duplicate contacts to a group based on config settings.</p>
  </blockquote>
  <blockquote>
  <p>If a group is selected for dedupe, optionally ext can list the duplicate row only when contact A and B are both part of the group.</p>
  </blockquote>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-28T19:42:26.290Z
Answers.LastActivityDate: 2024-05-28T19:42:26.290Z
Answers.UserDisplayName: petednz - fuzion
Answers.Body: <p>I would create a smart group based on contact-ids instead of names.</p>
  <p>The first time you run it, it will take long time anyway. (So probably good idea to split into smaller groups)</p>
  <p>But when you have done that, you could limit every new duplicate search to contact-ids over a certain number. In that way you only search the new ones. The old ones you have already deduped.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-03T17:10:48.737Z
Answers.LastActivityDate: 2024-06-03T17:10:48.737Z
Answers.UserDisplayName: Flying
Body: <p>We have a very large Civi database and are looking to dedupe. I have created smaller groups to work from (ex. all records with last name starting with &quot;A&quot;) but even that group has 12,000 records. It takes FOREVER to run the dedupe from that group against the whole database.</p>
  <p>Is there any way to limit the duplicate search so it only searches within the group?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-05-28T19:39:13.293Z
Comments.Text: we did some work for a client in similar position where we got the dedupe to run against contacts where both were in the same group. we also did it so the deduping ran on some cached data which also really sped things up. don't think we published but if Pradeep's suggestion doesn't help then come back to me and i will dig further
Comments.UserDisplayName: petednz - fuzion
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-05-29T06:58:33.197Z
Comments.Text: You could also use the extension X-Dedupe: https://docs.civicrm.org/xdedupe/en/latest/
  
  It has better performance when working with larger databases and quite a few nice features.
Comments.UserDisplayName: Fabian
Tags: <duplicate-contacts><dedupe-rules>
Title: Limit duplicate search
ThreadText: # Limit duplicate search
  
  (Asked by: username_0 on 2024-05-28)
  
  We have a very large Civi database and are looking to dedupe. I have created smaller groups to work from (ex. all records with last name starting with "A") but even that group has 12,000 records. It takes FOREVER to run the dedupe from that group against the whole database.
  
  Is there any way to limit the duplicate search so it only searches within the group?
  
  **Tags:** duplicate-contacts, dedupe-rules
  
  > **Comments:**
  > **username_1**: we did some work for a client in similar position where we got the dedupe to run against contacts where both were in the same group. we also did it so the deduping ran on some cached data which also really sped things up. don't think we published but if Pradeep's suggestion doesn't help then come back to me and i will dig further  
  > **username_2**: You could also use the extension X-Dedupe: https://docs.civicrm.org/xdedupe/en/latest/  It has better performance when working with larger databases and quite a few nice features.  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_3 on 2024-05-28. Score: 2
  
  Unfortunately, the dedupe screen provided by the CiviCRM searches contacts outside the group.
  
  But I have found [Deduper extension](https://civicrm.org/extensions/deduper) much useful for larger databases.
  
  
  ---
  
  ### Answer by username_1 on 2024-05-28. Score: 1
  
  You could also have a look at our extension [here](https://github.com/fuzionnz/nz.co.fuzion.dedupegroup)
  
  > Adds the potential duplicate contacts to a group based on config settings.
  
  > If a group is selected for dedupe, optionally ext can list the duplicate row only when contact A and B are both part of the group.
  
  
  ---
  
  ### Answer by username_4 on 2024-06-03. Score: 0
  
  I would create a smart group based on contact-ids instead of names.
  
  The first time you run it, it will take long time anyway. (So probably good idea to split into smaller groups)
  
  But when you have done that, you could limit every new duplicate search to contact-ids over a certain number. In that way you only search the new ones. The old ones you have already deduped.
  
  
  ---
  
  