Answers.Body: <p>Perhaps the best free explanation of CiviCRM's deduplication is from the CiviCRM blog last year (July 31, 2014): <a href="https://civicrm.org/blogs/spidersilk/understanding-civicrm-dedupe-rules" rel="noreferrer">Understanding CiviCRM Dedupe Rules</a></p>
  <p>For the whole process, I will let the author speak for themselves as it really is an excellent write up and worth reading in its entirety to understand the deduplication system.</p>
  <p>For this question, I suspect the issue comes down to the <em>usage</em> of the rule.</p>
  <p>Quoting the author of the blog post (from point 2):</p>
  <blockquote>
  <ul>
  <li><p><strong>Unsupervised rules</strong> are rules that are automatically checked when a user enters contact info into a form on the front end of the site (for example, when registering for an event, making a contribution, etc.). <em>If a match is found, CiviCRM automatically merges the new contact into the old one, without checking with the user</em>....</p>
  </li>
  <li><p><strong>Supervised rules</strong>, on the other hand, are rules that are automatically checked when an admin or staff member enters info into the back end of the site (i.e., the admin interface). <em>They do not automatically merge the contacts — they just trigger a pop-up suggesting that the contact may be a duplicate</em>....</p>
  </li>
  <li><p><strong>General rules</strong> are not triggered automatically at all. They can be run manually from the &quot;Find and Merge Duplicate Contacts&quot; page, or selected to run during a data import, but they do not ever just run on their own....</p>
  </li>
  </ul>
  </blockquote>
  <p>Rachel, based on your question, it looks as though your rule is setup correctly, but it is most likely set as a <em>general</em> rule, not the <em>unsupervised</em> rule for individuals. If you change the usage, I think you will find the results you're looking for.</p>
  <p>Side note: for perhaps the best paid explanation, <a href="http://civiteacher.com/" rel="noreferrer">Civiteacher.com</a> has excellent videos about <a href="http://civiteacher.com/tags/dupes" rel="noreferrer">CiviCRM deduplication</a>.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-09-15T06:24:02.643Z
Answers.LastActivityDate: 2015-09-15T06:42:48.510Z
Answers.LastEditDate: 2020-06-17T09:00:50.810Z
Answers.UserDisplayName: Allen Hutchison
Answers.Body: <p>You don't say what version of CiviCRM you are on, and you do say this is occurring on both contributions and on event registrations, but I will note here (for those happening onto this related issue) that there is a bug in CiviEvent, versions 4.3.4, 4.4.3, in which the unsupervised matching rule is not used in event registrations. See: <a href="https://issues.civicrm.org/jira/browse/CRM-14134">https://issues.civicrm.org/jira/browse/CRM-14134</a></p>
  
  <p>The fix for this is in version 4.5.  We are currently examining the fix to see if it can be applied to earlier versions.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2015-09-15T20:58:06.097Z
Answers.LastActivityDate: 2015-09-15T20:58:06.097Z
Answers.UserDisplayName: DaveD
Body: <p>My Dedupe rule is not working at all. I have it set to first name, last name, phone all with a weight of 10 and a weight threshold of 30. It appears like it is not working at all. I have many people that match on all 3 fields that are creating duplicates. This is happening on both contribution pages and event forms. Do I have the rule set up incorrectly or am I missing anything?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-09-14T18:45:10.273Z
Comments.Text: Hi Rachel, is your rule set as the unsupervised, supervised or general rule? Have you manually used the rule?
Comments.UserDisplayName: Allen Hutchison
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-09-15T00:42:32.077Z
Comments.Text: Rachel, I have the same problem with a similar new dedupe rule - first name 10, last name 10, though I used ZIP/Postal Code 10 rather than phone, weight threshold of 30. We're on Civi 4.6.8, Drupal 7.x . It actually caused our system to crash/kept running the process which had to be killed in the back end. Allen, I set up the rule as a General rule.
Comments.UserDisplayName: Joe McLaughlin
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-09-15T06:27:24.450Z
Comments.Text: Hi @JoeMcLaughlin, thank you for commenting.  Your comment prompted me to answer below.  The rule must be the unsupervised rule for individuals if you want the system to process your rule automatically.  See my answer for an explanation and link. Hope that helps!
Comments.UserDisplayName: Allen Hutchison
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-09-15T06:41:51.797Z
Comments.Text: Allen, thanks, but I was intentionally running it manually as General and it was processing endlessly and then had to be killed. So it failed. Rachel's issue may be the rule type but it may be similar to/the same as mine.
Comments.UserDisplayName: Joe McLaughlin
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-09-15T06:51:47.187Z
Comments.Text: Roger that, Joe.  One more idea, then I'll leave you alone :).  Can you run the rule "one group at a time" successfully?  http://wiki.civicrm.org/confluence/display/CRMDOC/Find+Duplicate+Contacts  Just trying to establish if it's the rule itself or the size of the job.  How many contacts are in your database?
Comments.UserDisplayName: Allen Hutchison
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-09-15T18:38:44.343Z
Comments.Text: This is the unsupervised rule, so it should work when contacts enter info into the contribution and event forms.
Comments.UserDisplayName: Rachel
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-09-15T19:24:12.393Z
Comments.Text: Hi @Rachel, I setup a test rule matching yours just now in my environment and created a duplicate of myself.  It appears that the phone field must be an exact match.  My first and last names were exact matches, but I made my dupe phone number (123) 456-7890 instead of 123-456-7890 (which is our setup's standard phone syntax). The rule did not catch my new entry as a dupe.  When I changed the weight threshold to 20, it caught the dupe.  It appears the phone field may be the culprit. How are phone numbers being entered in your system?
Comments.UserDisplayName: Allen Hutchison
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-09-15T20:55:08.013Z
Comments.Text: There is virtually no merging being done..even when the formatting of the phone numbers are exactly the same. It's almost as if the system does not think it is an unsupervised rule. Is there a way to change the rules back to the default initial setting or to verify the backend data is correct?
Comments.UserDisplayName: Rachel
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-09-16T14:43:17.950Z
Comments.Text: Rachel, when you use your rule manually, what happens? If you created a new unsupervised rule, you should be able to go to the original rule and mark it as unsupervised again. If you edited the original rule, then I would copy the rule components from the demo site for your CMS. What CMS and what version of Civi are you using?
Comments.UserDisplayName: Allen Hutchison
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2015-09-17T14:59:46.837Z
Comments.Text: When I do it manually it does work fine. I am using version 4.4.3
Comments.UserDisplayName: Rachel
Tags: <civievent><civicontribute><duplicate-contacts>
Title: Dedupe Rule not working
ThreadText: # Dedupe Rule not working
  
  (Asked by: username_0 on 2015-09-14)
  
  My Dedupe rule is not working at all. I have it set to first name, last name, phone all with a weight of 10 and a weight threshold of 30. It appears like it is not working at all. I have many people that match on all 3 fields that are creating duplicates. This is happening on both contribution pages and event forms. Do I have the rule set up incorrectly or am I missing anything?
  
  **Tags:** civievent, civicontribute, duplicate-contacts
  
  > **Comments:**
  > **username_1**: Hi username_0, is your rule set as the unsupervised, supervised or general rule? Have you manually used the rule?  
  > **username_2**: username_0, I have the same problem with a similar new dedupe rule - first name 10, last name 10, though I used ZIP/Postal Code 10 rather than phone, weight threshold of 30. We're on Civi 4.6.8, Drupal 7.x . It actually caused our system to crash/kept running the process which had to be killed in the back end. Allen, I set up the rule as a General rule.  
  > **username_1**: Hi @JoeMcLaughlin, thank you for commenting.  Your comment prompted me to answer below.  The rule must be the unsupervised rule for individuals if you want the system to process your rule automatically.  See my answer for an explanation and link. Hope that helps!  
  > **username_2**: Allen, thanks, but I was intentionally running it manually as General and it was processing endlessly and then had to be killed. So it failed. username_0's issue may be the rule type but it may be similar to/the same as mine.  
  > **username_1**: Roger that, Joe.  One more idea, then I'll leave you alone :).  Can you run the rule "one group at a time" successfully?  http://wiki.civicrm.org/confluence/display/CRMDOC/Find+Duplicate+Contacts  Just trying to establish if it's the rule itself or the size of the job.  How many contacts are in your database?  
  > **username_0**: This is the unsupervised rule, so it should work when contacts enter info into the contribution and event forms.  
  > **username_1**: Hi @username_0, I setup a test rule matching yours just now in my environment and created a duplicate of myself.  It appears that the phone field must be an exact match.  My first and last names were exact matches, but I made my dupe phone number (123) 456-7890 instead of 123-456-7890 (which is our setup's standard phone syntax). The rule did not catch my new entry as a dupe.  When I changed the weight threshold to 20, it caught the dupe.  It appears the phone field may be the culprit. How are phone numbers being entered in your system?  
  > **username_0**: There is virtually no merging being done..even when the formatting of the phone numbers are exactly the same. It's almost as if the system does not think it is an unsupervised rule. Is there a way to change the rules back to the default initial setting or to verify the backend data is correct?  
  > **username_1**: username_0, when you use your rule manually, what happens? If you created a new unsupervised rule, you should be able to go to the original rule and mark it as unsupervised again. If you edited the original rule, then I would copy the rule components from the demo site for your CMS. What CMS and what version of Civi are you using?  
  > **username_0**: When I do it manually it does work fine. I am using version 4.4.3  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2015-09-15. Score: 8
  
  Perhaps the best free explanation of CiviCRM's deduplication is from the CiviCRM blog last year (July 31, 2014): [Understanding CiviCRM Dedupe Rules](https://civicrm.org/blogs/spidersilk/understanding-civicrm-dedupe-rules)
  
  For the whole process, I will let the author speak for themselves as it really is an excellent write up and worth reading in its entirety to understand the deduplication system.
  
  For this question, I suspect the issue comes down to the *usage* of the rule.
  
  Quoting the author of the blog post (from point 2):
  
  > * **Unsupervised rules** are rules that are automatically checked when a user enters contact info into a form on the front end of the site (for example, when registering for an event, making a contribution, etc.). *If a match is found, CiviCRM automatically merges the new contact into the old one, without checking with the user*....
  > * **Supervised rules**, on the other hand, are rules that are automatically checked when an admin or staff member enters info into the back end of the site (i.e., the admin interface). *They do not automatically merge the contacts — they just trigger a pop-up suggesting that the contact may be a duplicate*....
  > * **General rules** are not triggered automatically at all. They can be run manually from the "Find and Merge Duplicate Contacts" page, or selected to run during a data import, but they do not ever just run on their own....
  
  username_0, based on your question, it looks as though your rule is setup correctly, but it is most likely set as a *general* rule, not the *unsupervised* rule for individuals. If you change the usage, I think you will find the results you're looking for.
  
  Side note: for perhaps the best paid explanation, [Civiteacher.com](http://civiteacher.com/) has excellent videos about [CiviCRM deduplication](http://civiteacher.com/tags/dupes).
  
  
  ---
  
  ### Answer by username_3 on 2015-09-15. Score: 5
  
  You don't say what version of CiviCRM you are on, and you do say this is occurring on both contributions and on event registrations, but I will note here (for those happening onto this related issue) that there is a bug in CiviEvent, versions 4.3.4, 4.4.3, in which the unsupervised matching rule is not used in event registrations. See: <https://issues.civicrm.org/jira/browse/CRM-14134>
  
  The fix for this is in version 4.5. We are currently examining the fix to see if it can be applied to earlier versions.
  
  
  ---
  
  