Page MenuHomePhabricator

Decide what to do about the increasing civi crm db size
Closed, ResolvedPublic

Description

See related epic.

Definition of done:

  • Everyone understands the situation
  • Do we know when we'll hit a critical point? Is it knowable? If so, when?
  • We have created new tasks for next steps in decreasing size

Details

Related Gerrit Patches:
wikimedia/fundraising/crm : masterStop audit logging on civicrm_omnimail_job_progress

Event Timeline

DStrine updated the task description. (Show Details)Jan 29 2020, 1:00 AM
Eileenmcnaughton added a subscriber: Eileenmcnaughton.

I'm working on my thoughts here https://docs.google.com/document/d/1PAHrjf1Z6iBrpqGR5xGQWW89c7aT1MZs7DogRerY3Kg/edit - only because I haven't figured out if there is anything in them I shouldn't post to phab

Change 571408 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Stop audit logging on civicrm_omnimail_job_progress

https://gerrit.wikimedia.org/r/571408

Change 571408 merged by Ejegg:
[wikimedia/fundraising/crm@master] Stop audit logging on civicrm_omnimail_job_progress

https://gerrit.wikimedia.org/r/571408

Eileenmcnaughton added a subscriber: Ejegg.

@Ejegg do you want to take a look at https://phabricator.wikimedia.org/T243870#5871061 - I will convert it into tasks once I have some feedback

Eileenmcnaughton added a subscriber: Dwisehaupt.EditedFeb 11 2020, 11:19 PM

Regarding "Do we know when we'll hit a critical point? Is it knowable? If so, when?" -

crunches

  • ability of frdev1001 to have both DBs on it - @Dwisehaupt reports a 3.6TB partition with about .6 of that available - a 15% growth would probably be a bit of a hard limit on this - but we are already past being able to restore without clearing out existing DB.
  • backup restore time - currently around 15 hours - but we could restore them separately without messing with integrity - we should probably have a phab task to figure out what this looks like before it happens if we want to rely on this - Dallas is also investigating another backup & restore method

@Dwisehaupt @Jeff_G @DStrine @Ejegg I've updated the document / response above and created a bunch of subtasks for the steps that I think we should take first. I haven't created a task for the review of backups / separate DB issue because I know Jeff & Dallas are already digging & perhaps there is a task.

We just deployed the 1st 3 & got DB reduction 933.5G to 819.8G

Jgreen moved this task from Triage to Watching on the fundraising-tech-ops board.Feb 19 2020, 10:43 PM
DStrine closed this task as Resolved.Feb 25 2020, 8:48 PM