Page MenuHomePhabricator

Enable dbctl for parsercache
Open, MediumPublic

Description

pc hosts are still pooled/depooled via MW and not using dbctl like the rest of the production MW databases.

We should integrate all the pc sections (pc1-pc4) into dbctl

Event Timeline

First we need to add them to dbctl, I'm not sure how that can be done. We currently have this in mw config:

                'parsercache-dbs' => [
                        'pc1' => '10.64.0.57',   # pc1011, A1 8.8TB 512GB # pc1
                        'pc2' => '10.64.16.65',  # pc1012, B1 8.8TB 512GB # pc2
                        'pc3' => '10.64.32.163', # pc1013, C5 8.8TB 512GB # pc3
                        'pc4' => '10.64.32.53',  # pc1016, C6 8.6TB 512GB # pc4
                        # spare: '10.64.48.89',  # pc1014, D6 8.8TB 512GB
                        # spare: '10.64.0.17',   # pc1015, A6 8.8TB 512GB
                        # Use spare(s) to replace any of the above if needed
                ],
and
                'parsercache-dbs' => [ 
                        'pc1' => '10.192.0.72',   # pc2011, A5 8.8TB 512GB # pc1 
                        'pc2' => '10.192.16.55',  # pc2012, B5 8.8TB 512GB # pc2
                        'pc3' => '10.192.32.57',  # pc2013, C1 8.8TB 512GB # pc3
                        'pc4' => '10.192.48.92',  # pc2016, D3 8.8TB 512GB # pc4
                        # spare: '10.192.48.52',  # pc2014, D1 8.8TB 512GB
                        # spare: '10.192.32.132', # pc2015, C5 8.8TB 512GB
                        # Use spare(s) to replace any of the above if needed
                ],

If there is a way to put this in externalLoads (don't need the spares), I can work with that. cc @Volans, @CDanis and @Scott_French

We do need to list the spares somehow somewhere because otherwise we will forget where they are (or even their hoatnames)

We do need to list the spares somehow somewhere because otherwise we will forget where they are (or even their hoatnames)

yeah, we probably need something like this then:

'10.192.0.72' => 'pc1',
'10.192.16.55' => 'pc2',
'10.192.48.52' => null // spare

Is the idea to start treating pc sections like everything else in dbctl or something more limited? (e.g., just managing the section to (singular) host mapping, as we do today in mediawiki-config)

If the former, I see that T316482 may have laid some of the groundwork for this kind of use case, but I also suppose this implies significant changes on the MW side (i.e., using the LBFactory for accessing pc sections, which AFAICT is not currently how this works).

Is the idea to start treating pc sections like everything else in dbctl or something more limited? (e.g., just managing the section to (singular) host mapping, as we do today in mediawiki-config)

I'd prefer it to be treated like any other section. Right now it is the only one which still needs to be done via MW, which makes it a snowflake and very painful (and time consuming) to depool/repool hosts.
Maybe we can treat it like x2, where we have a master and a couple of replicas as spares, that'd work for pcX too.

I think that treating them as x2 with "omit_replicas_in_mwconfig": true might just work. The spares could be either set as candidate masters for each section or just simple replicas for each section, given the above config. Upon promotion of a space to master in one section it should be removed from all the other sections and replaced with the new spare.

Volans triaged this task as Medium priority.Mon, Apr 29, 2:30 PM

If the former, I see that T316482 may have laid some of the groundwork for this kind of use case, but I also suppose this implies significant changes on the MW side (i.e., using the LBFactory for accessing pc sections, which AFAICT is not currently how this works).

Leave the mw side of things to me ^_^