Answers.Body: <p>I just used a general rule for org contact type import yesterday and it worked. </p>
  
  <p><a href="https://civicrm.stackexchange.com/questions/25137/import-contacts-creating-dupes-despite-dedupe-rules/25138#25138">Import Contacts creating dupes despite dedupe rules</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-12-05T17:15:05.780Z
Answers.LastActivityDate: 2018-12-05T17:15:05.780Z
Answers.UserDisplayName: Andy Burns
Answers.Body: <p>I have at least one customer that uses dedupe rules regularly in their processes and they work. They have different general rules like:</p>
  
  <ul>
  <li>Last name AND first name AND birth date</li>
  <li>Last name AND first name AND postal code</li>
  <li>E-mail OR (First Name AND Last Name AND Birth Date)</li>
  </ul>
  
  <p>Mind you, they do this deduping in the backend, so in CiviCRM. Their CiviCRM is on a different server than their public website.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-29T06:41:53.107Z
Answers.Text: To make it more specific: when webforms come in this customer uses the Extended Contact Matcher extension to find contacts. They have separated matching (finding the correct contact, we want to get all input from activists from our website and then organise the data ourselves) which they do from the website and deduping which they do regularly internally.
Answers.UserDisplayName: ErikH - CiviCooP
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-12-05T17:19:33.660Z
Answers.LastActivityDate: 2018-12-05T17:26:06.800Z
Answers.LastEditDate: 2018-12-05T17:26:06.800Z
Answers.UserDisplayName: ErikH - CiviCooP
Body: <p>I am having a problem with custom general rules for deduping.  Very inconsistent results that cause me to suspect that there may be significant issues.   One strange observation is that at times a general clearing of caches makes a difference.  But even that is not consistent and problematic if a rule is being used by a webform, for example.</p>
  
  <p>Are users have success with general rules? Failures? My test case is using First Name, Last Name and birth date. Is this a case where a general rewrite is needed and perhaps a Make It Happen campaign?</p>
  
  <p>Any insights would be appreciated.
  <a href="https://i.sstatic.net/8juox.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/8juox.png" alt="enter image description here"></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-12-05T19:09:10.267Z
Comments.Text: can you clarify your dedupe rule config ie the weighting and the threshold
Comments.UserDisplayName: petednz - fuzion
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-12-05T19:27:28.660Z
Comments.Text: sure - added to initial posting
Comments.UserDisplayName: Dave T
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-12-05T19:32:03.393Z
Comments.Text: I have now repeated failure > success cycle by clear Civi caches.   The need to flush cache is not clear as it is not consistently due to any of the things I have tested.   Matching via Webform is not working with this rule thus far even with cache clear.
Comments.UserDisplayName: Dave T
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-12-05T19:50:21.293Z
Comments.Text: Further testing - There is some sequence of events (not figured out yet) that seems to cause a need to clear cache to allow dedupe rule to be reset.
Comments.UserDisplayName: Dave T
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-12-05T21:01:24.637Z
Comments.Text: is there a reason for having values in the Length field? Dedupe may run faster without values in there
Comments.UserDisplayName: petednz - fuzion
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-12-06T09:19:38.843Z
Comments.Text: Agree with @petednz-fuzion - why the lengths?  I'm not sure what sense a length restriction makes on a birth date. I'd remove all the lengths.
Comments.UserDisplayName: Aidan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-12-31T01:50:59.647Z
Comments.Text: I've seen very strange behavior with dedupe rules since Civi 5.3. things like different results for different permission levels. Weight thresholds not being respected and all of the above not being consistent either
Comments.UserDisplayName: Yossi
Tags: <duplicate-contacts>
Title: Are General Deduping rules working?
ThreadText: # Are General Deduping rules working?
  
  (Asked by: username_0 on 2018-12-05)
  
  I am having a problem with custom general rules for deduping. Very inconsistent results that cause me to suspect that there may be significant issues. One strange observation is that at times a general clearing of caches makes a difference. But even that is not consistent and problematic if a rule is being used by a webform, for example.
  
  Are users have success with general rules? Failures? My test case is using First Name, Last Name and birth date. Is this a case where a general rewrite is needed and perhaps a Make It Happen campaign?
  
  Any insights would be appreciated.
  [![enter image description here](https://i.sstatic.net/8juox.png)](https://i.sstatic.net/8juox.png)
  
  **Tags:** duplicate-contacts
  
  > **Comments:**
  > **username_1**: can you clarify your dedupe rule config ie the weighting and the threshold  
  > **username_0**: sure - added to initial posting  
  > **username_0**: I have now repeated failure > success cycle by clear Civi caches.   The need to flush cache is not clear as it is not consistently due to any of the things I have tested.   Matching via Webform is not working with this rule thus far even with cache clear.  
  > **username_0**: Further testing - There is some sequence of events (not figured out yet) that seems to cause a need to clear cache to allow dedupe rule to be reset.  
  > **username_1**: is there a reason for having values in the Length field? Dedupe may run faster without values in there  
  > **username_2**: Agree with @username_1 - why the lengths?  I'm not sure what sense a length restriction makes on a birth date. I'd remove all the lengths.  
  > **username_3**: I've seen very strange behavior with dedupe rules since Civi 5.3. things like different results for different permission levels. Weight thresholds not being respected and all of the above not being consistent either  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_4 on 2018-12-05. Score: 1
  
  I just used a general rule for org contact type import yesterday and it worked.
  
  [Import Contacts creating dupes despite dedupe rules](https://civicrm.stackexchange.com/questions/25137/import-contacts-creating-dupes-despite-dedupe-rules/25138#25138)
  
  
  ---
  
  ### Answer by username_5 on 2018-12-05. Score: 1
  
  I have at least one customer that uses dedupe rules regularly in their processes and they work. They have different general rules like:
  
  * Last name AND first name AND birth date
  * Last name AND first name AND postal code
  * E-mail OR (First Name AND Last Name AND Birth Date)
  
  Mind you, they do this deduping in the backend, so in CiviCRM. Their CiviCRM is on a different server than their public website.
  
  > **Comments:**
  > **username_5**: To make it more specific: when webforms come in this customer uses the Extended Contact Matcher extension to find contacts. They have separated matching (finding the correct contact, we want to get all input from activists from our website and then organise the data ourselves) which they do from the website and deduping which they do regularly internally.  
  
  ---
  
  