Page MenuHomePhabricator

Database clean ups after migration of wikitech to production
Closed, ResolvedPublic

Description

Non exhaustive list of stuff to do:

  • Make wikitech use external storage.
  • Clean up text table of wikitech
  • Remove grants on s6 for old wikitech servers on puppet
  • Remove firewall holes on s6 for wikitech
  • Actually remove the grants from production
  • More?

Event Timeline

Change #1077000 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Wikitech: Connect wikitech to external storage

https://gerrit.wikimedia.org/r/1077000

Ladsgroup triaged this task as Medium priority.Oct 1 2024, 12:00 PM
Ladsgroup moved this task from Triage to In progress on the DBA board.

Change #1077000 merged by jenkins-bot:

[operations/mediawiki-config@master] Wikitech: Connect wikitech to external storage

https://gerrit.wikimedia.org/r/1077000

Mentioned in SAL (#wikimedia-operations) [2024-10-01T12:07:22Z] <ladsgroup@deploy2002> Started scap sync-world: Backport for [[gerrit:1077000|Wikitech: Connect wikitech to external storage (T376129)]]

Mentioned in SAL (#wikimedia-operations) [2024-10-01T12:09:38Z] <ladsgroup@deploy2002> ladsgroup: Backport for [[gerrit:1077000|Wikitech: Connect wikitech to external storage (T376129)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-10-01T12:17:15Z] <ladsgroup@deploy2002> Finished scap sync-world: Backport for [[gerrit:1077000|Wikitech: Connect wikitech to external storage (T376129)]] (duration: 09m 53s)

Mentioned in SAL (#wikimedia-operations) [2024-10-01T12:23:22Z] <Amir1> mwscript maintenance/storage/moveToExternal.php --wiki=labswiki --undo /home/ladsgroup/T376129.undo.sql DB cluster31 (T376129)

Change #1077082 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/puppet@production] mariadb: Remove specific wikitech grants

https://gerrit.wikimedia.org/r/1077082

Change #1077083 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/puppet@production] mariadb: Remove wikitech firewall holes

https://gerrit.wikimedia.org/r/1077083

Change #1077082 merged by Ladsgroup:

[operations/puppet@production] mariadb: Remove specific wikitech grants

https://gerrit.wikimedia.org/r/1077082

Change #1077083 merged by Ladsgroup:

[operations/puppet@production] mariadb: Remove wikitech firewall holes

https://gerrit.wikimedia.org/r/1077083

Change #1077359 had a related patch set uploaded (by Zabe; author: Zabe):

[operations/mediawiki-config@master] labswiki: Reduce revision-slots expiry to 60s

https://gerrit.wikimedia.org/r/1077359

Change #1077359 merged by jenkins-bot:

[operations/mediawiki-config@master] s6: Reduce revision-slots cache expiry to 60s

https://gerrit.wikimedia.org/r/1077359

Mentioned in SAL (#wikimedia-operations) [2024-10-02T12:05:28Z] <zabe@deploy2002> Started scap sync-world: Backport for [[gerrit:1077359|s6: Reduce revision-slots cache expiry to 60s (T183490 T376129)]]

Mentioned in SAL (#wikimedia-operations) [2024-10-02T12:09:06Z] <zabe@deploy2002> zabe: Backport for [[gerrit:1077359|s6: Reduce revision-slots cache expiry to 60s (T183490 T376129)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-10-02T12:14:19Z] <zabe@deploy2002> Finished scap sync-world: Backport for [[gerrit:1077359|s6: Reduce revision-slots cache expiry to 60s (T183490 T376129)]] (duration: 08m 50s)

Removed the grants, I'm going to run omg again to be sure.

Removed the grants, I'm going to run omg again to be sure.

Grants have been cleaned up.

Ladsgroup moved this task from In progress to Done on the DBA board.

I can't find anything else, there are probably more but I don't want to ticket to linger forever, if we find anything, we clean it up on the spot.

These tables probably should be dropped:

external_user ['labswiki']
ldap_domains ['labswiki']
oathauth_users_restore ['labswiki']
globalimagelinks ['labswiki', 'commonswiki', 'testcommonswiki']
oathauth_users ['labswiki', 'vewikimedia', 'foundationwiki']
oauth2_access_tokens ['labswiki', 'mediawikiwiki', 'metawiki']
oauth_accepted_consumer ['labswiki', 'mediawikiwiki', 'metawiki']
oauth_registered_consumer ['labswiki', 'mediawikiwiki', 'metawiki']
echo_email_batch ['labswiki', 'hywwiki', 'mediawikiwiki', 'officewiki', 'metawiki']
echo_event ['labswiki', 'hywwiki', 'mediawikiwiki', 'officewiki', 'metawiki']
echo_notification ['labswiki', 'hywwiki', 'mediawikiwiki', 'officewiki', 'metawiki']
echo_target_page ['labswiki', 'hywwiki', 'mediawikiwiki', 'officewiki', 'metawiki']

(from labswiki only)

I confirm this table hasn't been written into at least in the past three years:

root@db1173:/srv/sqldata/labswiki# ls -l | grep -i global
-rw-rw---- 1 mysql mysql       2867 Jul 22  2021 globalimagelinks.frm
-rw-rw---- 1 mysql mysql     131072 Jul 22  2021 globalimagelinks.ibd

(it's probably longer)