Customize query in gerrit
Details
Details
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Ottomata | T159170 Sunset MySQL data store for eventlogging | |||
| Resolved | Ottomata | T231858 Archive data on eventlogging MySQL to analytics replica before decomisioning | |||
| Resolved | elukey | T236818 Rerun sanitization before archiving eventlogging mysql data |
Event Timeline
Comment Actions
As part of this task I'd also clean up profile::mariadb::misc::eventlogging::sanitization from the db110[78] hosts :)
Script to run:
elukey@db1107:~$ sudo systemctl cat eventlogging_db_sanitization | grep ExecStart ExecStart=/usr/local/bin/eventlogging_cleaner --whitelist /srv/deployment/analytics/refinery/static_data/eventlogging/whitelist.yaml --yaml --older-than 90 --start-ts-file /srv/eventlogging_cleaner/eventlogging_cleaner --batch-size 10000 --sleep-between-batches 2 elukey@db1108:~$ sudo systemctl cat eventlogging_db_sanitization | grep ExecStart ExecStart=/usr/local/bin/eventlogging_cleaner --whitelist /srv/deployment/analytics/refinery/static_data/eventlogging/whitelist.yaml --yaml --older-than 90 --start-ts-file /srv/eventlogging_cleaner/eventlogging_cleaner --batch-size 10000 --sleep-between-batches 2
I'd also stop the timer/service beforehand to avoid issues.
Comment Actions
Change 548142 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] eventlogging: allow sanitization script to run on all db records
Comment Actions
High level plan that I have in mind:
- review/merge https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/548142/
- run sudo -u eventlogcleaner /usr/local/bin/eventlogging_cleaner --whitelist /srv/deployment/analytics/refinery/static_data/eventlogging/whitelist.yaml --yaml --older-than 0 --start-ts-file /srv/eventlogging_cleaner/eventlogging_cleaner --batch-size 10000 --sleep-between-batches 2
@mforns to triple check :)
Comment Actions
Change 548142 merged by Elukey:
[operations/puppet@production] eventlogging: allow sanitization script to run on all db records
Comment Actions
Change 548318 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] eventlogging: run sanitization script on all the db records
Comment Actions
Change 548318 merged by Elukey:
[operations/puppet@production] eventlogging: run sanitization script on all the db records
Comment Actions
Remaining steps:
- wait for the timer to complete to have full sanitization
- clean up sanitization stuff from db nodes
- clean up remaining puppet leftovers
Comment Actions
Change 549070 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::mariadb::misc::eventlogging::sanitization: ease clean up
Comment Actions
Change 549071 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] Remove Eventloggging sanitization automation from log databases
Comment Actions
Change 549070 merged by Elukey:
[operations/puppet@production] profile::mariadb::misc::eventlogging::sanitization: ease clean up
Comment Actions
Change 549071 merged by Elukey:
[operations/puppet@production] Remove Eventloggging sanitization automation from log databases
Comment Actions
Change 550738 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[analytics/refinery/scap@master] Remove db1107 and db1108 from scap targets
Comment Actions
Change 550738 merged by Ottomata:
[analytics/refinery/scap@master] Remove db1107 and db1108 from scap targets