- Contact for whom there is a single match in CiviCRM based on first name + last name + street address + city + postal code. This seems like the easiest & we can merge them civicrm/contact/view?reset=1&cid=490772
- Contact for whom there is a single match in CiviCRM based on first name + last name + email (same as #1)
- Contact has multiple matches based on the above. Option 1 - throw an exception & have someone go through & merge - probably after the first import this might be improved & perhaps then we would loosen to Option 2 - merge to the most recent contact who has source - 'check' civicrm/contact/view?reset=1&cid=19960248
|Open||None||T181090 Epic: Can the engage import dedupe on import?|
|Resolved||Eileenmcnaughton||T189617 Step 1 , 2 and 3 of engage import|
|Resolved||Eileenmcnaughton||T190119 Fix issues with Geocoder extension enabling disabled geocoders|
@LeanneS it seems there are a number of contacts in the engage file with first_name = Anonymous, last_name = Anonymous and no email. When that happens with Benevity we match them to the Anonymous contact AND we ignore any address data that may be associated with that contact. Are you happy for that handling for Engage too
@LeanneS a couple more questions - say we have an existing contact:
'first_name' => 'Minnie', 'last_name' => 'Mouse', 'contact_type' => 'Individual', 'email' => 'firstname.lastname@example.org', 'postal_code' => 98210, 'city' => 'Mouseville', 'street_address' => '35 Mousey Lane', 'location_type_id' => 'Home', ));
Is it possible that our new contact might be a match but have LESS address info - ie. the new contact has no city & in that case should we blank out the city?
I see a few error messages from my attempts: 'Error in line 5: (DB Error: already exists),' 'Unknown import error: Import aborted due to 10 consecutive errors, last error was at row 14: Could not delete entity id X,' 'Error in line 6: (Could not delete entity id X)'
@Eileenmcnaughton Thanks! I tested a couple new files and they overall seemed to import. I'm having some trouble using Find Contribs to check how they imported as it is very slow/won't load.
I'm curious how the records are merging for those with many duplicate records. For example, we have some monthly check donors in which the most recent March record merged with their October record, not February or their first record created. I'm trying to send a few examples, but the Recent Items links lead to different records than the names indicated.
I also noticed that the communications preferences aren't being reflected, such as do not mail.
@LeanneS I tested today on staging & got better results (civicrm/contact/view?reset=1&cid=25885282) - we were in a bit of a weird place last week with DB state so would be keen to test one more on staging before going live .
I have added another patch (https://gerrit.wikimedia.org/r/#/c/421803/) (check with me if it is deployed to staging before testing) to ensure the following fields are updated:
Do Not Email,Do Not Phone,Do Not SMS,Is Opt Out
(I left off 'Do Not Solicit' which is not in the files you sent me but added 'Do Not Mail' which is basically the same format as the above).
2 other questions from @Ejegg in review
- if we have 2 contacts with an email match & one is the most recent donor but the address is not their primary and the other has an older donation but it is their primary which do we match to? (ditto address)
- I am matching first_name='Anonymous' + last_name = 'Anonymous' + no email to the anonymous donor - is it possible that we might have contacts with no email & first_name = 'Anonymous' but an actual name for last name (if so, do we match to anonymous?)
- Do we need to handle empty first & last names or will there always be either a name or 'Anonymous' - the file I'm looking at has no empty names
- Go with most recent, make incoming the primary
- If we have anonymous+anonymous +address do not discard address, create new contact.
- conservative approach (skipping contacts with only first or last name OK for now)
- privacy field updates
- only matches to anon if all of email, street_address & postal code are blank (I figured just city/ state wasn't enough to create a new contact)?
- has handling for trailing . in street address
@Eileenmcnaughton it's been working great for me! For the last three imports I went back and double checked the file created with new contacts and did not find any that should have been merged - they were also pretty small to begin with though.
@LeanneS Is there anything else you think we should be testing before this gets closed?