Page MenuHomePhabricator

Delete deleted contacts merge activities
Closed, ResolvedPublic2 Estimate Story Points

Description

Almost all the activity data relates to 5 activity types.

TypeIDCount%
Contribution63310161233%
Email32313446623%
Contact Merged841722571717%
Contact Deleted by Merge871721536117%
Unsubscribe9985289339%

Of these:
Contribution - I’m doubtful about the value of having activities as well as the actual contributions
Email - we don’t know what the impact is of clearing out the email text per Elliott’s suggestion - we could experiment. It may be a lot or it might be the indexed fields doing the chewing
Contact Deleted by Merge - we would remove these in equal numbers to the contacts we delete in 2b). I wouldn’t advocate deleting these more than aggressively than that
Contact merged - we could consider removing these activities as they relate to contacts from 2b but I would not be rushing to do that.
Unsubscribe- I think we want to keep this activity type unculled

Details

Related Gerrit Patches:

Event Timeline

Jgreen moved this task from Triage to Watching on the fundraising-tech-ops board.Feb 19 2020, 10:41 PM

Digging into this I see our index sizes are

database_nametable_nameindex_namesize_in_mb
civicrmcivicrm_activityindex_activity_date_time843.00
civicrmcivicrm_activityindex_location779.00
civicrmcivicrm_activityUI_activity_type_id615.00
civicrmcivicrm_activityUI_source_record_id545.00
civicrmcivicrm_activityindex_is_deleted524.00
civicrmcivicrm_activityindex_is_current_revision524.00
civicrmcivicrm_activityFK_civicrm_activity_parent_id517.00
civicrmcivicrm_activityFK_civicrm_activity_campaign_id477.95
civicrmcivicrm_activityFK_civicrm_relationship_id477.95
civicrmcivicrm_activityFK_civicrm_activity_phone_id477.95
civicrmcivicrm_activityFK_civicrm_activity_original_id477.95
civicrmcivicrm_activityindex_medium_id477.95

Looking at those last suspiciously similar numbers

Original id - non null = 0, used for CiviCase

select * FROM civicrm_activity WHERE original_id IS NOT NULL;

Medium id -non null = 465, used for CiviCase but it looks like we had some early experimental use

Phone id non null = 0, unused in core???

relationship_id non null = 0 - used in CiviCase??

is_deleteed non null = 0 - used in CiviCase

@MBeat33 @NNichols @LeanneS @RLewis the activity table in our database is huge.

The question I have for you is how useful it is & how long seeing contributions in the activity log - is that something you make use of, or do you just use the contribution tab. Is it generally useful or only useful for a period. The same question applies to the activities recording that we sent an automated thank you email - do you care to know that after the first 6-12 months?

The Contributions tab is what DS uses to look up donations, and that tab also shows when an TY email for an individual donation was sent, so it's no loss for my team to not have those two types appear in the Activity Log.

Activity Types that are more useful to us include:

  1. Cancel Recurring Contribution
  2. Sent year-end summary receipt
  3. Contact Deleted by Merge
  4. Contact Merged

I feel like even though we have contributions logged on the contribution tab they are useful to have in the activities as it helps someone see a donor's timeline. As we start using activities to log our prospect stages we can see in one view when something important happens and when a contribution happens in relation to that. For example, we send an invite for an event and the donor turns around and makes a donation a few days later.

As for the automated TY emails - I can't see why we would need them since the Thank you date is on the contribution. @LeanneS @RLewis any thoughts here?

@NNichols is having that in the activity tab useful for older contributions as well? Is there a point it declines in usefulness?

Change 582954 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Remove activities relating to deleted-by-merge contacts.

https://gerrit.wikimedia.org/r/582954

I've put this in review to reflect that I have put one patch ^^ into review to remove all the contact-deleted-by-merge activities for deleted contacts

Change 582954 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Remove activities relating to deleted-by-merge contacts.

https://gerrit.wikimedia.org/r/582954

mepps added a subscriber: mepps.

Moving to Pending Deployment because the only patch on here is approved. Feel free to move back if I'm wrong!

@DStrine I've deleted a chunk of activities related to the contact delete but we should put this on the agenda for fortnightly/ DS

Currently sitting in 'done' because the patch I worked on is deployed but some discussion / potential spin-off yet to go

DStrine closed this task as Resolved.Tue, Mar 31, 8:08 PM
Eileenmcnaughton renamed this task from Cull low value data from activity table to Delete deleted contacts merge activities.Tue, Mar 31, 8:09 PM
Eileenmcnaughton set the point value for this task to 2.
Eileenmcnaughton set Final Story Points to 2.