Mentioned in SAL (#wikimedia-operations) [2024-04-19T21:03:30Z] <taavi> taavi@mwmaint1002 ~ $ mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=metawiki --logwiki=metawiki Kou.i5h 'Renamed user 8356771833137' # T362942

Mentioned in SAL (#wikimedia-operations) [2024-04-19T21:02:35Z] <taavi> taavi@mwmaint1002 ~ $ mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=eowiki --logwiki=metawiki 'Gzsimonfbi' 'Renamed user 2409354752759' # T362941

Mentioned in SAL (#wikimedia-operations) [2024-04-19T19:56:36Z] <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T362508, journal in uncertain state) xfer wikidata from wdqs2022.codfw.wmnet -> wdqs2023.codfw.wmnet w/ force delete existing files, repooling both afterwards

Mentioned in SAL (#wikimedia-operations) [2024-04-19T19:15:59Z] <ryankemper> [WDQS] T363004 Restarted wdqs2012 to clear out its in-application-memory ban lists (it had pybal's twisted user agent banned)

Mentioned in SAL (#wikimedia-operations) [2024-04-19T18:50:34Z] <ryankemper> [WDQS] T363004 Restarted wdqs2010 and wdqs2024 to clear out their in-application-memory ban lists

Mentioned in SAL (#wikimedia-operations) [2024-04-19T18:34:36Z] <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T362508, journal in uncertain state) xfer wikidata from wdqs2022.codfw.wmnet -> wdqs2023.codfw.wmnet w/ force delete existing files, repooling both afterwards

Mentioned in SAL (#wikimedia-operations) [2024-04-19T15:44:30Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1235 (T352010)', diff saved to https://phabricator.wikimedia.org/P61028 and previous config saved to /var/cache/conftool/dbconfig/20240419-154430-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-19T14:59:08Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1235 (T352010)', diff saved to https://phabricator.wikimedia.org/P61025 and previous config saved to /var/cache/conftool/dbconfig/20240419-145907-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-19T14:27:26Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1183 (T352010)', diff saved to https://phabricator.wikimedia.org/P61023 and previous config saved to /var/cache/conftool/dbconfig/20240419-142726-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-19T13:42:05Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1183 (T352010)', diff saved to https://phabricator.wikimedia.org/P61020 and previous config saved to /var/cache/conftool/dbconfig/20240419-134204-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-19T12:08:54Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1178 (T352010)', diff saved to https://phabricator.wikimedia.org/P61018 and previous config saved to /var/cache/conftool/dbconfig/20240419-120853-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-19T12:08:31Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1177 (T352010)', diff saved to https://phabricator.wikimedia.org/P61017 and previous config saved to /var/cache/conftool/dbconfig/20240419-120831-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-19T11:23:09Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1177 (T352010)', diff saved to https://phabricator.wikimedia.org/P61013 and previous config saved to /var/cache/conftool/dbconfig/20240419-112309-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-19T10:41:44Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1183 (T352010)', diff saved to https://phabricator.wikimedia.org/P61010 and previous config saved to /var/cache/conftool/dbconfig/20240419-104144-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-19T06:38:48Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1235 (T352010)', diff saved to https://phabricator.wikimedia.org/P60998 and previous config saved to /var/cache/conftool/dbconfig/20240419-063847-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-19T06:38:25Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1234 (T352010)', diff saved to https://phabricator.wikimedia.org/P60997 and previous config saved to /var/cache/conftool/dbconfig/20240419-063825-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-19T05:53:04Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1234 (T352010)', diff saved to https://phabricator.wikimedia.org/P60991 and previous config saved to /var/cache/conftool/dbconfig/20240419-055303-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-19T05:02:25Z] <marostegui> dbmaint Upgrade s7 eqiad to Bookworm and MariaDB 10.6 T362745

Mentioned in SAL (#wikimedia-operations) [2024-04-19T05:02:17Z] <marostegui> dbmaint Upgrade s7 codfw to Bookworm and MariaDB 10.6 T362745

Mentioned in SAL (#wikimedia-operations) [2024-04-18T23:42:51Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1177 (T352010)', diff saved to https://phabricator.wikimedia.org/P60986 and previous config saved to /var/cache/conftool/dbconfig/20240418-234247-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T23:42:25Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1172 (T352010)', diff saved to https://phabricator.wikimedia.org/P60985 and previous config saved to /var/cache/conftool/dbconfig/20240418-234225-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T22:57:03Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1172 (T352010)', diff saved to https://phabricator.wikimedia.org/P60982 and previous config saved to /var/cache/conftool/dbconfig/20240418-225702-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T22:31:40Z] <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) (T362508, excessive lag) xfer wikidata from wdqs2022.codfw.wmnet -> wdqs2023.codfw.wmnet w/ force delete existing files, repooling both afterwards

Mentioned in SAL (#wikimedia-operations) [2024-04-18T21:11:50Z] <bking@cumin2002> START - Cookbook sre.wdqs.data-transfer (T362508, excessive lag) xfer wikidata from wdqs2022.codfw.wmnet -> wdqs2023.codfw.wmnet w/ force delete existing files, repooling both afterwards

Mentioned in SAL (#wikimedia-cloud) [2024-04-18T21:03:24Z] <wmbot~bd808@tools-bastion-12> Stop & start to switch to buildservice managed image (T362288)

Mentioned in SAL (#wikimedia-operations) [2024-04-18T21:00:15Z] <cjming@deploy1002> Finished scap: Backport for [[gerrit:1020939|Add templateeditor right to sysops in dawiki and fix typo in group name (T361461)]] (duration: 16m 24s)

Mentioned in SAL (#wikimedia-cloud) [2024-04-18T20:59:53Z] <wmbot~bd808@tools-bastion-12> Built image for 4ca07721 (T362288)

Mentioned in SAL (#wikimedia-operations) [2024-04-18T20:46:20Z] <cjming@deploy1002> cjming and nmw03: Backport for [[gerrit:1020939|Add templateeditor right to sysops in dawiki and fix typo in group name (T361461)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-04-18T20:43:50Z] <cjming@deploy1002> Started scap: Backport for [[gerrit:1020939|Add templateeditor right to sysops in dawiki and fix typo in group name (T361461)]]

Mentioned in SAL (#wikimedia-operations) [2024-04-18T20:42:53Z] <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on wdqs2023.codfw.wmnet with reason: T362508

Mentioned in SAL (#wikimedia-operations) [2024-04-18T20:42:44Z] <bking@cumin2002> START - Cookbook sre.hosts.downtime for 1:00:00 on wdqs2023.codfw.wmnet with reason: T362508

Mentioned in SAL (#wikimedia-operations) [2024-04-18T20:42:32Z] <cjming@deploy1002> Finished scap: Backport for [[gerrit:1021518|Temporarily restore wgMinervaApplyKnownTemplateHacks for cached HTML (T362747)]] (duration: 17m 14s)

Mentioned in SAL (#wikimedia-operations) [2024-04-18T20:32:57Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1234 (T352010)', diff saved to https://phabricator.wikimedia.org/P60980 and previous config saved to /var/cache/conftool/dbconfig/20240418-203256-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T20:32:34Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1232 (T352010)', diff saved to https://phabricator.wikimedia.org/P60979 and previous config saved to /var/cache/conftool/dbconfig/20240418-203234-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T20:27:45Z] <cjming@deploy1002> jdlrobson and cjming: Backport for [[gerrit:1021518|Temporarily restore wgMinervaApplyKnownTemplateHacks for cached HTML (T362747)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-04-18T20:25:17Z] <cjming@deploy1002> Started scap: Backport for [[gerrit:1021518|Temporarily restore wgMinervaApplyKnownTemplateHacks for cached HTML (T362747)]]

Mentioned in SAL (#wikimedia-operations) [2024-04-18T19:52:45Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2217 (T361627)', diff saved to https://phabricator.wikimedia.org/P60976 and previous config saved to /var/cache/conftool/dbconfig/20240418-195244-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T19:47:11Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1232 (T352010)', diff saved to https://phabricator.wikimedia.org/P60975 and previous config saved to /var/cache/conftool/dbconfig/20240418-194711-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T19:07:22Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2217 (T361627)', diff saved to https://phabricator.wikimedia.org/P60972 and previous config saved to /var/cache/conftool/dbconfig/20240418-190722-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T19:03:07Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2217 (T361627)', diff saved to https://phabricator.wikimedia.org/P60971 and previous config saved to /var/cache/conftool/dbconfig/20240418-190249-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T19:02:27Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2214 (T361627)', diff saved to https://phabricator.wikimedia.org/P60970 and previous config saved to /var/cache/conftool/dbconfig/20240418-190226-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T18:46:46Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1232 (T352010)', diff saved to https://phabricator.wikimedia.org/P60968 and previous config saved to /var/cache/conftool/dbconfig/20240418-184645-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T18:46:23Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1228 (T352010)', diff saved to https://phabricator.wikimedia.org/P60967 and previous config saved to /var/cache/conftool/dbconfig/20240418-184623-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T18:27:33Z] <dancy@deploy1002> rebuilt and synchronized wikiversions files: group2 wikis to 1.43.0-wmf.1 refs T361395

Mentioned in SAL (#wikimedia-operations) [2024-04-18T18:17:05Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2214 (T361627)', diff saved to https://phabricator.wikimedia.org/P60964 and previous config saved to /var/cache/conftool/dbconfig/20240418-181704-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T18:14:50Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2214 (T361627)', diff saved to https://phabricator.wikimedia.org/P60962 and previous config saved to /var/cache/conftool/dbconfig/20240418-181450-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T18:10:49Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2193 (T361627)', diff saved to https://phabricator.wikimedia.org/P60961 and previous config saved to /var/cache/conftool/dbconfig/20240418-181048-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T18:01:00Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1228 (T352010)', diff saved to https://phabricator.wikimedia.org/P60960 and previous config saved to /var/cache/conftool/dbconfig/20240418-180059-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T17:25:26Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2193 (T361627)', diff saved to https://phabricator.wikimedia.org/P60957 and previous config saved to /var/cache/conftool/dbconfig/20240418-172525-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T17:24:12Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2193 (T361627)', diff saved to https://phabricator.wikimedia.org/P60956 and previous config saved to /var/cache/conftool/dbconfig/20240418-172412-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T17:23:49Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2180 (T361627)', diff saved to https://phabricator.wikimedia.org/P60955 and previous config saved to /var/cache/conftool/dbconfig/20240418-172349-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T16:38:27Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2180 (T361627)', diff saved to https://phabricator.wikimedia.org/P60951 and previous config saved to /var/cache/conftool/dbconfig/20240418-163827-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T16:36:20Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2180 (T361627)', diff saved to https://phabricator.wikimedia.org/P60950 and previous config saved to /var/cache/conftool/dbconfig/20240418-163612-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T16:36:01Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2169 (T361627)', diff saved to https://phabricator.wikimedia.org/P60949 and previous config saved to /var/cache/conftool/dbconfig/20240418-163600-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T15:50:39Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2169 (T361627)', diff saved to https://phabricator.wikimedia.org/P60946 and previous config saved to /var/cache/conftool/dbconfig/20240418-155038-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T15:45:49Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2169 (T361627)', diff saved to https://phabricator.wikimedia.org/P60945 and previous config saved to /var/cache/conftool/dbconfig/20240418-154547-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T15:45:25Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2158 (T361627)', diff saved to https://phabricator.wikimedia.org/P60944 and previous config saved to /var/cache/conftool/dbconfig/20240418-154524-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T15:12:56Z] <claime> Pooling and uncordoning mw1355.eqiad.wmnet,mw1480.eqiad.wmnet,mw1481.eqiad.wmnet,mw1487.eqiad.wmnet - T351074

Mentioned in SAL (#wikimedia-operations) [2024-04-18T15:04:39Z] <claime> Running homer 'cr*eqiad*' commit 'T351074'

Mentioned in SAL (#wikimedia-operations) [2024-04-18T15:00:03Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2158 (T361627)', diff saved to https://phabricator.wikimedia.org/P60941 and previous config saved to /var/cache/conftool/dbconfig/20240418-150003-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T14:55:13Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2158 (T361627)', diff saved to https://phabricator.wikimedia.org/P60940 and previous config saved to /var/cache/conftool/dbconfig/20240418-145512-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T14:54:36Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2151 (T361627)', diff saved to https://phabricator.wikimedia.org/P60939 and previous config saved to /var/cache/conftool/dbconfig/20240418-145435-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T14:12:40Z] <claime> Depooling mw1355.eqiad.wmnet,mw1480.eqiad.wmnet,mw1481.eqiad.wmnet,mw1487.eqiad.wmnet - T351074

Mentioned in SAL (#wikimedia-operations) [2024-04-18T14:11:32Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 Finished scap: Backport for [[gerrit:1019779|Added extendedconfirmed and templateeditor rights to dawiki (T361461)]] (duration: 16m 51s)

Mentioned in SAL (#wikimedia-operations) [2024-04-18T14:09:14Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2151 (T361627)', diff saved to https://phabricator.wikimedia.org/P60936 and previous config saved to /var/cache/conftool/dbconfig/20240418-140913-marostegui.json

Mentioned in SAL (#wikimedia-analytics) [2024-04-18T14:07:28Z] <btullis> restarted the hadoop-yarn-resourcemanager.service on an-master100[3-4] to pick up new queue settings for T361499

Mentioned in SAL (#wikimedia-operations) [2024-04-18T14:04:23Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2151 (T361627)', diff saved to https://phabricator.wikimedia.org/P60935 and previous config saved to /var/cache/conftool/dbconfig/20240418-140421-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T14:03:59Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2129 (T361627)', diff saved to https://phabricator.wikimedia.org/P60934 and previous config saved to /var/cache/conftool/dbconfig/20240418-140359-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T13:57:20Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 nmw03 and lucaswerkmeister-wmde: Backport for [[gerrit:1019779|Added extendedconfirmed and templateeditor rights to dawiki (T361461)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-04-18T13:54:40Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 Started scap: Backport for [[gerrit:1019779|Added extendedconfirmed and templateeditor rights to dawiki (T361461)]]

Mentioned in SAL (#wikimedia-operations) [2024-04-18T13:51:56Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 Finished scap: Backport for [[gerrit:1015151|Add 'mainpage-title-loggedin' to $wgForceUIMsgAsContentMsg (T361171)]] (duration: 19m 37s)

Mentioned in SAL (#wikimedia-operations) [2024-04-18T13:47:43Z] <jynus> add grants for dbprov1005 at dbbackups (m1) T362509

Mentioned in SAL (#wikimedia-operations) [2024-04-18T13:37:16Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 lucaswerkmeister-wmde and jhsoby: Backport for [[gerrit:1015151|Add 'mainpage-title-loggedin' to $wgForceUIMsgAsContentMsg (T361171)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-04-18T13:32:19Z] <logmsgbot> lucaswerkmeister-wmde@deploy1002 Started scap: Backport for [[gerrit:1015151|Add 'mainpage-title-loggedin' to $wgForceUIMsgAsContentMsg (T361171)]]

Mentioned in SAL (#wikimedia-operations) [2024-04-18T13:18:38Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2129 (T361627)', diff saved to https://phabricator.wikimedia.org/P60928 and previous config saved to /var/cache/conftool/dbconfig/20240418-131836-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T13:13:11Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2129 (T361627)', diff saved to https://phabricator.wikimedia.org/P60927 and previous config saved to /var/cache/conftool/dbconfig/20240418-131311-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T13:12:48Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2124 (T361627)', diff saved to https://phabricator.wikimedia.org/P60926 and previous config saved to /var/cache/conftool/dbconfig/20240418-131248-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T12:54:15Z] <sukhe> sudo cumin -b1 -s600 "A:dnsbox" "systemctl restart ntp.service" to pick up magru /24: T346722

Mentioned in SAL (#wikimedia-operations) [2024-04-18T12:49:15Z] <elukey> move aqs codfw cassandra instances to PKI TLS certs - T352647

Mentioned in SAL (#wikimedia-operations) [2024-04-18T12:27:24Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2124 (T361627)', diff saved to https://phabricator.wikimedia.org/P60919 and previous config saved to /var/cache/conftool/dbconfig/20240418-122721-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T12:22:28Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2124 (T361627)', diff saved to https://phabricator.wikimedia.org/P60918 and previous config saved to /var/cache/conftool/dbconfig/20240418-122227-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T12:15:59Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1231 (T361627)', diff saved to https://phabricator.wikimedia.org/P60916 and previous config saved to /var/cache/conftool/dbconfig/20240418-121559-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T11:52:03Z] <claime> Pooling and uncordoning mw2302.codfw.wmnet,mw2303.codfw.wmnet,mw2304.codfw.wmnet,mw2332.codfw.wmnet,mw2333.codfw.wmnet,mw2334.codfw.wmnet - T351074

Mentioned in SAL (#wikimedia-operations) [2024-04-18T11:42:04Z] <claime> Running homer 'cr*codfw*' commit 'T351074'

Mentioned in SAL (#wikimedia-analytics) [2024-04-18T11:41:33Z] <btullis> adding new 'launchers' yarn queue and renaming 'fifo' to 'gpus' for T361499

Mentioned in SAL (#wikimedia-operations) [2024-04-18T11:30:37Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1231 (T361627)', diff saved to https://phabricator.wikimedia.org/P60913 and previous config saved to /var/cache/conftool/dbconfig/20240418-113037-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T11:28:27Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1231 (T361627)', diff saved to https://phabricator.wikimedia.org/P60912 and previous config saved to /var/cache/conftool/dbconfig/20240418-112827-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T11:29:19Z] <cgoubert@deploy1002> Finished scap: Redeploy mw-on-k8s with full rebuild - Fix setting php.timeout - T358308 (duration: 37m 04s)

Mentioned in SAL (#wikimedia-operations) [2024-04-18T11:28:17Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1172 (T352010)', diff saved to https://phabricator.wikimedia.org/P60911 and previous config saved to /var/cache/conftool/dbconfig/20240418-112816-ladsgroup.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T11:25:01Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1224 (T361627)', diff saved to https://phabricator.wikimedia.org/P60910 and previous config saved to /var/cache/conftool/dbconfig/20240418-112459-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T11:11:33Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1183 (T355609)', diff saved to https://phabricator.wikimedia.org/P60909 and previous config saved to /var/cache/conftool/dbconfig/20240418-111132-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T10:52:15Z] <cgoubert@deploy1002> Started scap: Redeploy mw-on-k8s with full rebuild - Fix setting php.timeout - T358308

Mentioned in SAL (#wikimedia-operations) [2024-04-18T10:45:15Z] <claime> Rebuild php7.4-fpm production images - T358308

Mentioned in SAL (#wikimedia-operations) [2024-04-18T10:39:34Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1224 (T361627)', diff saved to https://phabricator.wikimedia.org/P60904 and previous config saved to /var/cache/conftool/dbconfig/20240418-103933-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T10:34:22Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1224 (T361627)', diff saved to https://phabricator.wikimedia.org/P60903 and previous config saved to /var/cache/conftool/dbconfig/20240418-103422-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T10:34:00Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1201 (T361627)', diff saved to https://phabricator.wikimedia.org/P60902 and previous config saved to /var/cache/conftool/dbconfig/20240418-103359-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T10:26:10Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1183 (T355609)', diff saved to https://phabricator.wikimedia.org/P60901 and previous config saved to /var/cache/conftool/dbconfig/20240418-102609-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T10:25:38Z] <claime> Depooling mw2302.codfw.wmnet,mw2303.codfw.wmnet,mw2304.codfw.wmnet,mw2332.codfw.wmnet,mw2333.codfw.wmnet,mw2334.codfw.wmnet for reimage - T351074

Mentioned in SAL (#wikimedia-operations) [2024-04-18T10:18:42Z] <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1183 (T355609)', diff saved to https://phabricator.wikimedia.org/P60899 and previous config saved to /var/cache/conftool/dbconfig/20240418-101841-marostegui.json

Mentioned in SAL (#wikimedia-operations) [2024-04-18T09:53:31Z] <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1228 (T352010)', diff saved to https://phabricator.wikimedia.org/P60897 and previous config saved to /var/cache/conftool/dbconfig/20240418-095331-ladsgroup.json