Page MenuHomePhabricator

Fix disk capacity issues on civi1002
Closed, ResolvedPublic

Description

We have a few civicrm/audit related things that heading toward overrunning available disk space on civi1002. We have about 300GB available on /srv and are at 76% capacity.

/var/lib/silverpop_export (26GB)

This overran /var today so I moved it to /srv where we have more free space. This is related to T342879.

/srv/archive/civi/audit (101GB)

There are audit files going back to 2014, can some of this be deleted or compressed?

Event Timeline

Jgreen renamed this task from Move silverpop_export data directory from /var/lib to /srv to Fix disk capacity issues on civi1002.Jul 30 2023, 10:48 AM
Jgreen removed a project: Fundraising-Backlog.
Jgreen updated the task description. (Show Details)

Whoops, I put the wrong tag on this when I created it!

It seems fr-tech has the only interest in retaining the audit files, and we don't need them for that long. Perhaps we start only keeping them for 2 years and see how that goes. We could possibly trim it back even further in the future, if needed.

Dwisehaupt subscribed.

Adding the fr-tech-ops tag on this one. We may be able to use the archive purge script to enforce this setting on the civi hosts.

Jgreen changed the task status from Open to Stalled.EditedNov 7 2023, 4:33 PM
Jgreen moved this task from Up Next to Blocked on the fundraising-tech-ops board.

I don't see how we can do the 2Y expiration without changing the way we store these files. We would need date stamps in the filenames or reliable file create times (currently they start 3/15/2023).

Dwisehaupt claimed this task.
Dwisehaupt moved this task from Blocked to Done on the fundraising-tech-ops board.

This has been largely addressed by not syncing some debug log files that were not required. Closing.

commit fab6bd22efd50e883689f14c8c58ff18b91fb9e1
Date:   Thu Apr 25 14:51:24 2024 -0400

    exclude payments.* keeping payments-* from archive push to civi for audit processing