Page MenuHomePhabricator

Upgrade and rebuild s1
Closed, ResolvedPublic

Description

Let's proactively rebuild s1, for the following tables: linter recentchanges pagelinks

  • dbstore1008
  • db2216
  • db2212
  • db2203
  • db2202
  • db2188
  • db2186
  • db2176
  • db2174
  • db2173
  • db2170
  • db2153
  • db2146
  • db2145
  • db2141
  • db1251
  • db1240
  • db1239
  • db1235
  • db1234
  • db1232
  • db1219
  • db1218
  • db1207
  • db1206
  • db1196
  • db1195
  • db1186
  • db1184
  • db1169
  • db1163
  • db1154
  • clouddb1017
  • clouddb1013
  • an-redacteddb1001

Event Timeline

Marostegui triaged this task as Medium priority.
Marostegui moved this task from Triage to In progress on the DBA board.

Mentioned in SAL (#wikimedia-operations) [2025-02-06T13:13:01Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1235 db2188 T385561', diff saved to https://phabricator.wikimedia.org/P73313 and previous config saved to /var/cache/conftool/dbconfig/20250206-131300-marostegui.json

Icinga downtime and Alertmanager silence (ID=ff8dde3f-0be3-4fc6-b41a-29b432f4329a) set by root@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db2188.codfw.wmnet

Icinga downtime and Alertmanager silence (ID=1cafc0b8-aa6e-4c49-a791-048369baeb62) set by root@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db1235.eqiad.wmnet

Icinga downtime and Alertmanager silence (ID=146bc76e-69fa-40a7-8674-8050581ccdfd) set by root@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db2145.codfw.wmnet

Icinga downtime and Alertmanager silence (ID=b3e2d583-45ff-4606-a061-de3945e8ba20) set by root@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db1234.eqiad.wmnet

Icinga downtime and Alertmanager silence (ID=0cc004ca-c02a-4993-88f1-0b4c51e7ee9d) set by root@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db1232.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2025-02-12T15:27:39Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2176 T385561', diff saved to https://phabricator.wikimedia.org/P73446 and previous config saved to /var/cache/conftool/dbconfig/20250212-152738-marostegui.json

Icinga downtime and Alertmanager silence (ID=e1adabf0-8594-48e3-ae7b-771cc668aa30) set by marostegui@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db2176.codfw.wmnet

Icinga downtime and Alertmanager silence (ID=7ca60113-4a74-4598-a754-2e2da5548187) set by marostegui@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db1219.eqiad.wmnet

Icinga downtime and Alertmanager silence (ID=0bc7c017-209d-44dc-a6f9-4ff31ea96279) set by marostegui@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db1186.eqiad.wmnet

Icinga downtime and Alertmanager silence (ID=fd8d4eb6-9307-483c-89e9-4be4189075ae) set by marostegui@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db1195.eqiad.wmnet

Icinga downtime and Alertmanager silence (ID=5c07daae-926d-4a3b-96d9-e50eb4a17a11) set by marostegui@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db2153.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2025-02-26T11:39:36Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1206 db2170 T385561', diff saved to https://phabricator.wikimedia.org/P73674 and previous config saved to /var/cache/conftool/dbconfig/20250226-113935-root.json

Icinga downtime and Alertmanager silence (ID=487682e7-e7ed-4fae-b0c7-0cb4f3b71666) set by marostegui@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db2170.codfw.wmnet

Icinga downtime and Alertmanager silence (ID=7090d63a-beaf-4771-ba80-91cc21d5e7c6) set by marostegui@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db1206.eqiad.wmnet

Icinga downtime and Alertmanager silence (ID=bb149c8e-73e0-4fee-af0f-ba562e49e35e) set by marostegui@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db2174.codfw.wmnet

Icinga downtime and Alertmanager silence (ID=f98cb467-bd3a-4885-a9b6-64a25c3c14e5) set by marostegui@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db1207.eqiad.wmnet

Icinga downtime and Alertmanager silence (ID=6fee8ce7-f09a-436e-803c-59c258bb0e04) set by marostegui@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db2186.codfw.wmnet

Icinga downtime and Alertmanager silence (ID=0299e1ef-38b9-4b95-8ab0-9c4bc76cef48) set by marostegui@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db1196.eqiad.wmnet

Icinga downtime and Alertmanager silence (ID=a7215ebb-65ea-4969-9d27-1b08d2ec3466) set by root@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db2173.codfw.wmnet

Icinga downtime and Alertmanager silence (ID=f7af46a6-43cf-4a48-bf55-a5d0ba3e8c31) set by marostegui@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db1218.eqiad.wmnet

Icinga downtime and Alertmanager silence (ID=77c25323-bac2-4640-ac6f-1aa8eb59adf6) set by marostegui@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db1184.eqiad.wmnet

Icinga downtime and Alertmanager silence (ID=e158d092-1f1a-4fb6-99c5-e14c40e461d3) set by marostegui@cumin1002 for 12:00:00 on 1 host(s) and their services with reason: Index rebuild

db2212.codfw.wmnet
Marostegui updated the task description. (Show Details)

This is done - db2212 will start repooling automatically once the lag is gone