Page MenuHomePhabricator

Contacts with contributions being deleted
Closed, ResolvedPublic

Description

We had previously talked about this issue - contact records being deleted (automatically?) that have contributions.
T205941

To find these records: Find Contacts - Advanced search
Criteria:
"Search in Trash (deleted contacts)"
Contributions: Date = Last 2 year including today

There are 208 contacts in this search.

Event Timeline

NNichols created this task.Mar 11 2019, 9:15 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 11 2019, 9:15 PM

Ouch - confusing timeline

We have 2 types of date fields in Civi

  • Date fields
  • Timestamp fields

The latter records dates internally as GMT & renders in user timezone - so the Created: May 11th, 2018 8:11 PM is accurate - it's just been converted to my timezone.

I'm a bit less convinced the merge happened before the contribution was added but haven't yet figured that out

I note that 26362994 & 26363503 were merged at May 11th, 2018 3:02 AM

The most recent of these is

2018-05-11 09:44:09

I think that we fixed some job scheduling a while back to prevent any clashes so I think this breaks down into the following

  1. fix entries where merged contacts still have records
  2. follow up - add some sort of job that runs integrity queries for things that have hit us in the past & emails failmail if some exist
  3. investigate a locking mechanism per https://phabricator.wikimedia.org/T212749 to make our concurrent processing more robust & prevent more deadlocks which should prevent incomplete data rollback

Change 496092 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm/civicrm@master] Fix contact.merge api to pass check_permissions parameter through to the deeper layer

https://gerrit.wikimedia.org/r/496092

Change 496092 merged by jenkins-bot:
[wikimedia/fundraising/crm/civicrm@master] Fix contact.merge api to pass check_permissions parameter through to the deeper layer

https://gerrit.wikimedia.org/r/496092

@LeanneS @NNichols We've done this restore - a search for deleted with contributions still unearths a few - maybe we could take a look at these at fortnightly - they seemed like mostly old or odd ones to me but we could just undelete them all?

Examples discussed in Fortnightly:
4970326
14644918

We first enabled logging (before we started deduping) on 2016-05-02

This contact was already deleted at that point 14644918 - the 2 entries on her change log that appear to be deletes are actually updates to her address (geocoding) I think that because she is deleted they are being incorrectly displayed as deletes on the change log (we can log a follow on issue or ignore that). I found another version of the contact (14644918) with an equivalent donation on the same day - the contact is also dead. I would be inclined to delete the donation on the deleted contact it looks like a fairly intentional action & since we've dug into it we probably want to ensure we never do again....

Looking at the other now

So contact 4970326

  • her recent activity is the fact we unsubscribed her based on Silverpop information. Oddly the activity is not showing up on her activity tab. We could perhaps log a phab to check why but I would wait until after the next civi update before digging into that
  • she was merged in 2017

So my impression that she was in some way a recent data issue was wrong - I think the confusingness in the UI for deleted contacts was to blame for that - per earlier comment

So what to do with her? This contact was deleted (not as part of a merge) by Sandra in donor services on 2017-11-28 20:00:25 - she also changed her to be on hold at the same time.

I'm not quite sure what to do with this contact - but like the last one - it's not indicative of some wider issue

@LeanneS long essay on the 2 contacts ^^

I added phabs for the 2 things I noticed
https://phabricator.wikimedia.org/T219257
https://phabricator.wikimedia.org/T219258

(probably unless you want to encourage their progress through the system they will be outcompeted for sprint-attention)

@LeanneS did you dig anymore into this. Just wondering if we should keep it open

Eileenmcnaughton closed this task as Resolved.Apr 23 2019, 9:25 PM