Page MenuHomePhabricator

Remove groups from db configs
Closed, ResolvedPublic

Description

Per T239453#5910993 we should look at removing these groups from the MW db lb configs:

  • contributions
  • logpager
  • recentchanges
  • recentchangeslinked
  • watchlist

That would leave us with:

  • api
  • dump
  • vslow

This would greatly simplify automating depooling/repooling db instances.

Outcome

  • Confirmation and understanding that these configs are functionally safe to remove in productrion without requiring any code changes, even if they are still referenced. To be walk through by by @Krinkle together with one or two people from PET as part of knowledge transfer and familiarity with MediaWiki-libs-Rdbms.
  • Understanding and agreement on which of these (if any) we need to keep, and why. To be carried out by Platform Engineering based on the history of the db index and its usage in code.
  • T267077: Document remaining database load groups
  • Remove obsolete db group parameters from MW core and WMF-deployed extensions.

Removal progress

  • s1
    • eqiad
    • codfw
  • s2
    • eqiad
    • codfw
  • s3
    • eqiad
    • codfw
  • s4
    • eqiad
    • codfw
  • s5
    • eqiad
    • codfw
  • s6
    • eqiad
    • codfw
  • s7
    • eqiad
    • codfw
  • s8
    • eqiad
    • codfw

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Mentioned in SAL (#wikimedia-operations) [2022-01-13T07:50:12Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove recentchangeslinked group from s7 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P18715 and previous config saved to /var/cache/conftool/dbconfig/20220113-075012-marostegui.json

recentchangeslinked removed from s7 eqiad

Mentioned in SAL (#wikimedia-operations) [2022-01-13T08:39:24Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove recentchanges group from s7 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P18717 and previous config saved to /var/cache/conftool/dbconfig/20220113-083923-marostegui.json

recentchanges removed from s7 eqiad. There are no more groups on s7. Going to see if I need to re-adjust weights.

Mentioned in SAL (#wikimedia-operations) [2022-01-13T12:41:41Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove all special groups from s3 codfw T263127', diff saved to https://phabricator.wikimedia.org/P18729 and previous config saved to /var/cache/conftool/dbconfig/20220113-124140-marostegui.json

I have removed all special groups from s3 codfw. Slowly starting in eqiad too now.

Mentioned in SAL (#wikimedia-operations) [2022-01-13T12:43:00Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove contributions group from s3 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P18730 and previous config saved to /var/cache/conftool/dbconfig/20220113-124300-marostegui.json

contributions removed from s3 eqiad

Mentioned in SAL (#wikimedia-operations) [2022-01-14T06:35:54Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove logpager group from s3 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P18735 and previous config saved to /var/cache/conftool/dbconfig/20220114-063554-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-01-17T10:48:01Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove recentchangeslinked group from s3 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P18761 and previous config saved to /var/cache/conftool/dbconfig/20220117-104801-marostegui.json

recentchangeslinked removed from s3 eqiad

Mentioned in SAL (#wikimedia-operations) [2022-01-17T13:45:20Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove recentchanges group from s3 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P18762 and previous config saved to /var/cache/conftool/dbconfig/20220117-134520-marostegui.json

recentchanges removed from s3 eqiad

Mentioned in SAL (#wikimedia-operations) [2022-01-18T05:46:59Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove watchlist group from s3 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P18764 and previous config saved to /var/cache/conftool/dbconfig/20220118-054659-marostegui.json

watchlist removed from s3 eqiad - no more groups remaining in s3!

Mentioned in SAL (#wikimedia-operations) [2022-01-19T06:53:19Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Adjust s3 weights T263127', diff saved to https://phabricator.wikimedia.org/P18796 and previous config saved to /var/cache/conftool/dbconfig/20220119-065318-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-01-24T11:52:37Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove special groups from s8 codfw T263127', diff saved to https://phabricator.wikimedia.org/P19037 and previous config saved to /var/cache/conftool/dbconfig/20220124-115236-marostegui.json

s8 codfw groups removed entirely.

Mentioned in SAL (#wikimedia-operations) [2022-01-24T11:53:35Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove contributions from s8 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P19038 and previous config saved to /var/cache/conftool/dbconfig/20220124-115334-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-01-25T14:26:15Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove logpager from s8 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P19165 and previous config saved to /var/cache/conftool/dbconfig/20220125-142614-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-01-26T06:46:54Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove recentchangeslinked from s8 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P19233 and previous config saved to /var/cache/conftool/dbconfig/20220126-064653-marostegui.json

s8 recentchangeslinked eqiad removed

Mentioned in SAL (#wikimedia-operations) [2022-01-26T13:26:00Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove recentchanges from s8 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P19315 and previous config saved to /var/cache/conftool/dbconfig/20220126-132600-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-01-27T07:05:57Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove watchlist from s8 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P19380 and previous config saved to /var/cache/conftool/dbconfig/20220127-070557-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-01-31T06:11:22Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove logpager from s4 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P19572 and previous config saved to /var/cache/conftool/dbconfig/20220131-061121-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-02-01T10:25:13Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove contributions from s4 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P19782 and previous config saved to /var/cache/conftool/dbconfig/20220201-102512-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-02-01T10:41:19Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove all special groups from s4 codfw T263127', diff saved to https://phabricator.wikimedia.org/P19785 and previous config saved to /var/cache/conftool/dbconfig/20220201-104118-marostegui.json

Removed all groups from s4 codfw.

Mentioned in SAL (#wikimedia-operations) [2022-02-02T10:38:31Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove recentchanges and recentchanges groups from s4 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P19910 and previous config saved to /var/cache/conftool/dbconfig/20220202-103830-marostegui.json

s4 recentchanges recentchangeslinked eqiad removed

Mentioned in SAL (#wikimedia-operations) [2022-02-04T08:20:10Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove watchlist group from s4 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P20164 and previous config saved to /var/cache/conftool/dbconfig/20220204-082010-marostegui.json

s4 watchlist eqiad removed. s4 has no more groups

Mentioned in SAL (#wikimedia-operations) [2022-02-04T11:04:28Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove all special groups from s1 codfw T263127', diff saved to https://phabricator.wikimedia.org/P20171 and previous config saved to /var/cache/conftool/dbconfig/20220204-110427-marostegui.json

Removed all groups from s1 codfw.

Mentioned in SAL (#wikimedia-operations) [2022-02-08T06:03:10Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove contributions group from s1 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P20236 and previous config saved to /var/cache/conftool/dbconfig/20220208-060310-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-02-09T07:35:28Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove logpager group from s1 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P20410 and previous config saved to /var/cache/conftool/dbconfig/20220209-073528-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-02-10T05:49:11Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove recentchangeslinked group from s1 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P20451 and previous config saved to /var/cache/conftool/dbconfig/20220210-054911-marostegui.json

s1 recentchangeslinked eqiad removed

Mentioned in SAL (#wikimedia-operations) [2022-02-10T09:27:27Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove recentchanges group from s1 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P20492 and previous config saved to /var/cache/conftool/dbconfig/20220210-092727-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2022-02-11T09:02:24Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Remove watchlist group from s1 eqiad T263127', diff saved to https://phabricator.wikimedia.org/P20599 and previous config saved to /var/cache/conftool/dbconfig/20220211-090223-marostegui.json

s1 watchlist eqiad removed

All done, no more special groups in production (apart from api and vslow,dump)

While inspecting an anon edit on dewiki (s5) I noticed the post request for the edit established two separate replica connections. The second one for a query tagged as contributions. This task's solution would presumably avoid that extra connection.

I analyzed our backend save timing dashboard around the times the largest wikis in terms of edit volume (wikidata, enwiki) had their eqiad contributions group removed.

In respective breakdowns by content type (wikidata entity vs wikipedia content), no noticable change was found. To the extent that it would show up, it would have affected the lower percentiles (p50) most since the extra connection was a fixed cost.

Screenshot 2022-02-15 at 13.43.38.png (1×2 px, 309 KB)

Screenshot 2022-02-15 at 13.48.09.png (1×2 px, 326 KB)

Change 825343 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/core@master] api: Deprecate ApiQuery::getNamedDB and ::selectNamedDB()

https://gerrit.wikimedia.org/r/825343

Change 825343 merged by jenkins-bot:

[mediawiki/core@master] api: Deprecate ApiQuery::getNamedDB and ::selectNamedDB()

https://gerrit.wikimedia.org/r/825343