Page MenuHomePhabricator

Rebuild logging table on s1 hosts
Closed, DeclinedPublic

Description

There are some weirdness with the optimizer on the query for T304626 - we need to rebuild the logging table on enwiki to refresh the optimizer stats so it picks up the right index.
Not all the hosts have this issue, so this is the list of hosts we need to rebuild the table on:

Some of them in codfw, but let's run it with replication on the master so we don't have to do them one by one

  • codfw
  • db1184
  • db1169
  • db1164
  • db1163
  • db1154 (sanitarium - will get it via replication)
  • db1135
  • db1119
  • db1118 (master)
  • db1106 (sanitarium master to be run with replication)
  • clouddb* (will get it via replication)

Related Objects

Event Timeline

Marostegui moved this task from Triage to In progress on the DBA board.

Change 774371 had a related patch set uploaded (by Marostegui; author: Marostegui):

[operations/puppet@production] db1106,db1154: Disable notifications

https://gerrit.wikimedia.org/r/774371

Change 774371 merged by Marostegui:

[operations/puppet@production] db1106,db1154: Disable notifications

https://gerrit.wikimedia.org/r/774371

Mentioned in SAL (#wikimedia-operations) [2022-03-28T08:47:05Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1169 T304812', diff saved to https://phabricator.wikimedia.org/P23373 and previous config saved to /var/cache/conftool/dbconfig/20220328-084705-marostegui.json

so db1169 was rebuilt and the optimizer didn't really change its plan, so this might not be enough.

db1169 hasn't changed its plan after a rebuild, optimizer and analyze. We need to solve T304626 in code somehow