Page MenuHomePhabricator

examine the possibility of purging data from civicrm.civicrm_activity
Closed, ResolvedPublic

Description

We're looking at various ways to counter growth of the civicrm database, and civicrm.civicrm_activity is a >100GB table. We may be able to selectively purge data from this table.

Tests:

Get an order of magnitude on the impact of purging data from column 'details'

  • procedure:
    • create table civicrm_activity_test like civicrm_activity;
    • insert into civicrm_activity_test select * from civicrm_activity;
    • check size civicrm_activity_test.ibd (result: 104GB)
    • update civicrm_activity_test set details=null;
    • create table civicrm_activity_test2 like civicrm_activity_test;
    • insert into civicrm_activity_test2 select * from civicrm_activity_test;
    • check size of civicrm_activity_test2.ibd (result: 25GB)

Event Timeline

Jgreen added subscribers: Eileenmcnaughton, Ejegg.

@Eileenmcnaughton @Ejegg result of deleting all the civicrm.civicrm_activity 'details' column reduces the table from 104GB to 25GB, see task updated description for details.

@Jgreen so I'm going with this being worth more work :-) did log_civicrm_activity have an even larger change?

@Jgreen so I'm going with this being worth more work :-) did log_civicrm_activity have an even larger change?

log_civicrm_activity is a much smaller table, only 37GB before nulling that column

TY emails over 1 year old were purged on 2021-05-24 reclaiming 53G from civicrm.civicrm_activity and 22G from civicrm.log_civicrm_activity.

Jgreen claimed this task.

Looks like this was investigated and partially completed.