Page MenuHomePhabricator

Make sre.mysql.depool work with esX
Closed, ResolvedPublic

Description

sre.mysql.depool (and .pool) handle pcX and msX should also work with esX

external store has RO sections and RW sections and they have different cases:

RO section:
if a replica goes down, we simply depool via dbctl instance esXX depool
if a master goes down, we simply should "switch" masters in dbctl (we obviously need to pick a replica within the same section):

dbctl -s $DC section esX set-master esXXXX
dbctl instance esXXXX depool 
dbctl config commit -m "Your commit message XXX"

RW section:
if a replica goes down, we simply depool via dbctl instance esXX depool
if a master goes down, we can remove writes from that section (reads would remain unaffected)

sudo dbctl --scope eqiad section esX ro "Depooling"
sudo dbctl --scope codfw section esX ro "Depooling"
sudo dbctl config commit -m "Master esX crashed "

(Note: for RW sections we do have to run those two commands regardless of the active dc)

That would allow the replicas to receive reads but no writes.

How to find if it is a writable section may be the tricky part (as they all are RW on dbctl due to MW reasons, but RO ones aren't written) - for now maybe it can be just hardcoded (we spawn RW sections every 5 years or so).
Currently the RO ones: es1 es2 es3 es4 es5

The writable ones are on https://noc.wikimedia.org/conf/highlight.php?file=db-production.php at the bottom but however, we do not automatically translate cluster32 to esX anywhere other than the table names.
At the moment they are: es6 es7

Event Timeline

Marostegui moved this task from Triage to Refine on the DBA board.

To be discussed on Monday's meeting.

Marostegui renamed this task from Make sre.mysql.depool work with RW esX to Make sre.mysql.depool work with esX.Mon, Jul 13, 1:29 PM
Marostegui updated the task description. (Show Details)
FCeratto-WMF changed the task status from Open to In Progress.Tue, Jul 14, 2:17 PM
FCeratto-WMF claimed this task.

Change #1311405 had a related patch set uploaded (by Federico Ceratto; author: Federico Ceratto):

[operations/cookbooks@master] sre.mysql.pool: support esX sections

https://gerrit.wikimedia.org/r/1311405

Change #1311427 had a related patch set uploaded (by Federico Ceratto; author: Federico Ceratto):

[operations/puppet@production] management.yaml: Add python3-pydantic package to cumin

https://gerrit.wikimedia.org/r/1311427

Change #1311427 merged by Federico Ceratto:

[operations/puppet@production] management.yaml: Add python3-pydantic package to cumin

https://gerrit.wikimedia.org/r/1311427

Completed depooling of es2051 by fceratto@cumin1003: Test

Starting pool of es2051 by fceratto@cumin1003: Test

Completed pooling of es2051 by fceratto@cumin1003: Test

Completed depooling of es1055 by marostegui@cumin1003: testing

Completed Setting section es1 as read-write by marostegui@cumin1003: testing

Starting pool of es1055 by marostegui@cumin1003: testing

Completed depooling of es2051 by marostegui@cumin1003: testing

Completed Setting section es1 as read-write by marostegui@cumin1003: testing

Starting pool of es2051 by marostegui@cumin1003: testing

Completed pooling of es1055 by marostegui@cumin1003: testing

Completed depooling of es1054 by marostegui@cumin1003: testing

Starting pool of es1054 by marostegui@cumin1003: testing

Completed pooling of es2051 by marostegui@cumin1003: testing

Completed depooling of es1038 by marostegui@cumin1003: testing

Completed Setting section es6 as read-write by marostegui@cumin1003: testing

Starting pool of es1038 by marostegui@cumin1003: testing

Completed pooling of es1054 by marostegui@cumin1003: testing

Completed pooling of es1038 by marostegui@cumin1003: testing

Completed depooling of es1037 by marostegui@cumin1003: testing

Completed Setting section es6 as read-write by marostegui@cumin1003: testing

Starting pool of es1037 by marostegui@cumin1003: testing

Completed pooling of es1037 by marostegui@cumin1003: testing

Tests done:

Tests:

RO
section es1

  • Depooled es2051 this triggered the wrong message T430769#12148904
  • Pooled es2051

section es3

  • Depooled master es1054 (correctly picked es1057 and left es1054 depooled)
  • Repooled old master es1054 as a replica

RW
section es6

replica operations:

  • Depooled es1038 - this seems to have logged the wrong message, see: T430769#12149138
  • Repooled es1038

Master operations

Completed fake-switchover from es1050 to new master es1055 by marostegui@cumin1003: testing

Completed depool es6 replica es1038 by marostegui@cumin1003: testing

Completed setting read-write section es7 as read-only by marostegui@cumin1003: testing

Completed Setting section es7 as read-write by marostegui@cumin1003: testing

Starting pool of es1035 by marostegui@cumin1003: testing

Completed pooling of es1035 by marostegui@cumin1003: testing

es1 depool worked fine, no weird message.
es6 replica depool worked fine too , no weird message.
es7 master depool and repool worked fine too

Change #1311405 merged by jenkins-bot:

[operations/cookbooks@master] sre.mysql.pool: support esX sections

https://gerrit.wikimedia.org/r/1311405

FCeratto-WMF moved this task from Refine to Done on the DBA board.