Page MenuHomePhabricator

Stashbot (Stashbot)
Bot that likes to stash things away for laterBot

Projects

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Friday

  • Clear sailing ahead.

User Details

User Since
Feb 10 2016, 12:29 AM (334 w, 15 h)
Roles
Bot
Availability
Available
IRC Nick
stashbot
LDAP User
Stashbot
MediaWiki User
Unknown

Recent Activity

Today

Stashbot added a comment to T310546: Recable cloudcephosd1021 from cloudsw1-c8-eqiad to cloudsw2-c8-eqiad.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T14:55:59Z] <cmjohnson1> moving switch ports cloudcephosd1021 from cloudsw1-c to cloudsw2-c T310546

Wed, Jul 6, 2:56 PM · SRE, cloud-services-team (Hardware), ops-eqiad, DC-Ops
Stashbot added a comment to T305845: Re-import full planet data into codfw.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T14:22:43Z] <akosiaris> depool eqiad kartotherian T305845

Wed, Jul 6, 2:22 PM · serviceops, Patch-For-Review, Maps
Stashbot added a comment to T305845: Re-import full planet data into codfw.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T14:17:14Z] <akosiaris> pool codfw for kartotherian T305845

Wed, Jul 6, 2:17 PM · serviceops, Patch-For-Review, Maps
Stashbot added a comment to T301878: Send score to eventgate when requested.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T13:53:21Z] <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:810007|Add a new Eventgate stream for revision-score events (T301878)]] (duration: 03m 46s)

Wed, Jul 6, 1:53 PM · Patch-For-Review, Epic, Lift-Wing, Machine-Learning-Team (Active Tasks)
Stashbot added a comment to T311106: Investigate mariadb 10.6 performance regression during spikes/high load.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T13:17:15Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depool db1132 (T311106)', diff saved to https://phabricator.wikimedia.org/P30930 and previous config saved to /var/cache/conftool/dbconfig/20220706-131715-ladsgroup.json

Wed, Jul 6, 1:17 PM · Sustainability (Incident Followup), SRE-OnFire, DBA
Stashbot added a comment to T311106: Investigate mariadb 10.6 performance regression during spikes/high load.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T08:26:03Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'db1132 (re)pooling @ 100%: Test done (T311106)', diff saved to https://phabricator.wikimedia.org/P30898 and previous config saved to /var/cache/conftool/dbconfig/20220706-082603-ladsgroup.json

Wed, Jul 6, 8:26 AM · Sustainability (Incident Followup), SRE-OnFire, DBA
Stashbot added a comment to T311106: Investigate mariadb 10.6 performance regression during spikes/high load.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T08:25:40Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 100%: Test done (T311106)', diff saved to https://phabricator.wikimedia.org/P30897 and previous config saved to /var/cache/conftool/dbconfig/20220706-082540-ladsgroup.json

Wed, Jul 6, 8:25 AM · Sustainability (Incident Followup), SRE-OnFire, DBA
Stashbot added a comment to T308072: 1.39.0-wmf.19 deployment blockers.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T08:12:12Z] <jnuche@deploy1002> Synchronized php: group1 wikis to 1.39.0-wmf.19 refs T308072 (duration: 03m 39s)

Wed, Jul 6, 8:12 AM · Patch-For-Review, Release-Engineering-Team (Priority Backlog 📥), Release, Train Deployments
Stashbot added a comment to T311106: Investigate mariadb 10.6 performance regression during spikes/high load.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T08:10:59Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'db1132 (re)pooling @ 75%: Test done (T311106)', diff saved to https://phabricator.wikimedia.org/P30893 and previous config saved to /var/cache/conftool/dbconfig/20220706-081059-ladsgroup.json

Wed, Jul 6, 8:11 AM · Sustainability (Incident Followup), SRE-OnFire, DBA
Stashbot added a comment to T311106: Investigate mariadb 10.6 performance regression during spikes/high load.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T08:10:36Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 75%: Test done (T311106)', diff saved to https://phabricator.wikimedia.org/P30892 and previous config saved to /var/cache/conftool/dbconfig/20220706-081036-ladsgroup.json

Wed, Jul 6, 8:10 AM · Sustainability (Incident Followup), SRE-OnFire, DBA
Stashbot added a comment to T308072: 1.39.0-wmf.19 deployment blockers.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T08:08:32Z] <jnuche@deploy1002> rebuilt and synchronized wikiversions files: group1 wikis to 1.39.0-wmf.19 refs T308072

Wed, Jul 6, 8:08 AM · Patch-For-Review, Release-Engineering-Team (Priority Backlog 📥), Release, Train Deployments
Stashbot added a comment to T311106: Investigate mariadb 10.6 performance regression during spikes/high load.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T07:55:55Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'db1132 (re)pooling @ 25%: Test done (T311106)', diff saved to https://phabricator.wikimedia.org/P30888 and previous config saved to /var/cache/conftool/dbconfig/20220706-075555-ladsgroup.json

Wed, Jul 6, 7:56 AM · Sustainability (Incident Followup), SRE-OnFire, DBA
Stashbot added a comment to T311106: Investigate mariadb 10.6 performance regression during spikes/high load.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T07:55:33Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 25%: Test done (T311106)', diff saved to https://phabricator.wikimedia.org/P30887 and previous config saved to /var/cache/conftool/dbconfig/20220706-075532-ladsgroup.json

Wed, Jul 6, 7:55 AM · Sustainability (Incident Followup), SRE-OnFire, DBA
Stashbot added a comment to T311106: Investigate mariadb 10.6 performance regression during spikes/high load.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T07:40:52Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'db1132 (re)pooling @ 10%: Test done (T311106)', diff saved to https://phabricator.wikimedia.org/P30882 and previous config saved to /var/cache/conftool/dbconfig/20220706-074051-ladsgroup.json

Wed, Jul 6, 7:41 AM · Sustainability (Incident Followup), SRE-OnFire, DBA
Stashbot added a comment to T311106: Investigate mariadb 10.6 performance regression during spikes/high load.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T07:40:29Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'db1135 (re)pooling @ 10%: Test done (T311106)', diff saved to https://phabricator.wikimedia.org/P30881 and previous config saved to /var/cache/conftool/dbconfig/20220706-074028-ladsgroup.json

Wed, Jul 6, 7:40 AM · Sustainability (Incident Followup), SRE-OnFire, DBA
Stashbot added a comment to T311106: Investigate mariadb 10.6 performance regression during spikes/high load.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T07:30:53Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depool db1135, if anything breaks, it's marostegui's fault (T311106)', diff saved to https://phabricator.wikimedia.org/P30880 and previous config saved to /var/cache/conftool/dbconfig/20220706-073052-ladsgroup.json

Wed, Jul 6, 7:31 AM · Sustainability (Incident Followup), SRE-OnFire, DBA
Stashbot added a comment to T312177: mediawiki.org Main Page issue on mobile.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T07:20:42Z] <ladsgroup@deploy1002> Synchronized php-1.39.0-wmf.19/extensions/DiscussionTools/modules/dt.init.less: Backport: [[gerrit:811406|Revert "Hide the lede section on mobile when DiscussionTools is enabled" (T312177)]] (duration: 03m 37s)

Wed, Jul 6, 7:20 AM · Editing-team (FY2021-22 Kanban Board), DiscussionTools, MinervaNeue, MobileFrontend
Stashbot added a comment to T312162: Adjust the field type of cx_notification_log.cxn_date/cxn_newest to fixed binary on wmf wikis.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T05:48:58Z] <marostegui> dbmaint x1@eqiad T312162

Wed, Jul 6, 5:49 AM · Schema-change-in-production, DBA
Stashbot added a comment to T312162: Adjust the field type of cx_notification_log.cxn_date/cxn_newest to fixed binary on wmf wikis.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T05:48:55Z] <marostegui> dbmaint s3@eqiad T312162

Wed, Jul 6, 5:48 AM · Schema-change-in-production, DBA
Stashbot added a comment to T312161: Adjust the field type of cx_lists.cxl_start_time/cxl_end_time to fixed binary on wmf wikis.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T05:46:12Z] <marostegui> dbmaint s3@eqiad T312161

Wed, Jul 6, 5:46 AM · Schema-change-in-production, DBA
Stashbot added a comment to T312161: Adjust the field type of cx_lists.cxl_start_time/cxl_end_time to fixed binary on wmf wikis.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T05:45:56Z] <marostegui> dbmaint x1@eqiad T312161

Wed, Jul 6, 5:45 AM · Schema-change-in-production, DBA
Stashbot added a comment to T311493: Productionize db2153.codfw.wmnet - db2174.codfw.wmnet.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T05:10:46Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Add db2159 to dbctl T311493', diff saved to https://phabricator.wikimedia.org/P30869 and previous config saved to /var/cache/conftool/dbconfig/20220706-051046-marostegui.json

Wed, Jul 6, 5:11 AM · Patch-For-Review, DBA
Stashbot added a comment to T310662: Acceptably efficient AbuseFilter profiling storage backend.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T04:18:46Z] <tstarling@deploy1002> Synchronized php-1.39.0-wmf.19/extensions/AbuseFilter: T310662 deployment with possible post-send error spike due to ServiceWiring/FilterProfiler interdependency (duration: 03m 33s)

Wed, Jul 6, 4:18 AM · Patch-For-Review, Sustainability (MediaWiki-MultiDC), AbuseFilter, Performance Issue
Stashbot added a comment to T310662: Acceptably efficient AbuseFilter profiling storage backend.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T03:34:39Z] <tstarling@deploy1002> Finished scap: WRStats core prereq T310662 g811407 (duration: 17m 20s)

Wed, Jul 6, 3:34 AM · Patch-For-Review, Sustainability (MediaWiki-MultiDC), AbuseFilter, Performance Issue
Stashbot added a comment to T310662: Acceptably efficient AbuseFilter profiling storage backend.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T03:17:18Z] <tstarling@deploy1002> Started scap: WRStats core prereq T310662 g811407

Wed, Jul 6, 3:17 AM · Patch-For-Review, Sustainability (MediaWiki-MultiDC), AbuseFilter, Performance Issue
Stashbot added a comment to T310662: Acceptably efficient AbuseFilter profiling storage backend.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T02:30:16Z] <tstarling@deploy1002> Synchronized wmf-config/InitialiseSettings.php: T310662 g 811394 harmless prerequisite (duration: 03m 39s)

Wed, Jul 6, 2:30 AM · Patch-For-Review, Sustainability (MediaWiki-MultiDC), AbuseFilter, Performance Issue
Stashbot added a comment to T274463: Backups for GitLab.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T01:21:25Z] <mutante> gitlab1004 rm /lib/systemd/system/rsync-data-backup-gitlab2001.wikimedia.org.* ; systemctl reset-failed (T274463, T307142) - fix icinga alert after gitlab2001 was decom'ed, we didn't have puppet remove the timer/service

Wed, Jul 6, 1:21 AM · serviceops-collab, GitLab (Infrastructure), serviceops, Data-Persistence-Backup, Patch-For-Review, User-brennen
Stashbot added a comment to T307142: bring new gitlab hardware servers into production.

Mentioned in SAL (#wikimedia-operations) [2022-07-06T01:21:25Z] <mutante> gitlab1004 rm /lib/systemd/system/rsync-data-backup-gitlab2001.wikimedia.org.* ; systemctl reset-failed (T274463, T307142) - fix icinga alert after gitlab2001 was decom'ed, we didn't have puppet remove the timer/service

Wed, Jul 6, 1:21 AM · Patch-For-Review, GitLab (Infrastructure), serviceops

Yesterday

Stashbot added a comment to T309648: Restore lost index in cloudelastic.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T23:15:38Z] <ryankemper@cumin1001> END (PASS) - Cookbook sre.elasticsearch.rolling-operation (exit_code=0) Operation.RESTART (2 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - ryankemper@cumin1001 - T309648

Tue, Jul 5, 11:15 PM · Patch-For-Review, Discovery-Search (Current work)
Stashbot added a comment to T309648: Restore lost index in cloudelastic.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T22:48:44Z] <ryankemper@cumin1001> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (2 nodes at a time) for ElasticSearch cluster cloudelastic: cloudelastic cluster restart - ryankemper@cumin1001 - T309648

Tue, Jul 5, 10:48 PM · Patch-For-Review, Discovery-Search (Current work)
Stashbot added a comment to T309648: Restore lost index in cloudelastic.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T22:28:22Z] <ryankemper> T309648 Manually restarting cloudelastic1006 before proceeding to a normal rolling restart of cloudelastic

Tue, Jul 5, 10:28 PM · Patch-For-Review, Discovery-Search (Current work)
Stashbot added a comment to T309648: Restore lost index in cloudelastic.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T22:28:11Z] <ryankemper> T309648 Manually restarting cloudelastic1006 before proceeding to a normal rolling restart of cloudelasti

Tue, Jul 5, 10:28 PM · Patch-For-Review, Discovery-Search (Current work)
Stashbot added a comment to T311773: [Layout] Deploy title/tab order everywhere (Make it so that the title should always be above tabs).

Mentioned in SAL (#wikimedia-operations) [2022-07-05T21:55:25Z] <ebernhardson@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:811339|Enable title above tabs everywhere (T311773)]] (duration: 03m 23s)

Tue, Jul 5, 9:55 PM · Patch-For-Review, Readers-Web-Backlog (Kanbanana-2022-23-Q1), Desktop Improvements
Stashbot added a comment to T309648: Restore lost index in cloudelastic.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T21:35:20Z] <ebernhardson@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: Revert: [[gerrit:811350|cirrus: Disable commonswiki writes to cloudelastic (T309648)]] (duration: 03m 42s)

Tue, Jul 5, 9:35 PM · Patch-For-Review, Discovery-Search (Current work)
Stashbot added a comment to T309648: Restore lost index in cloudelastic.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T21:27:36Z] <ebernhardson@deploy1002> Synchronized php-1.39.0-wmf.18/extensions/CirrusSearch/includes/Job/ElasticaWrite.php: Backport: [[gerrit:811279|job queue: Squelch errors related to unwritable cloudelastic (T309648)]] (duration: 03m 37s)

Tue, Jul 5, 9:27 PM · Patch-For-Review, Discovery-Search (Current work)
Stashbot added a comment to T309648: Restore lost index in cloudelastic.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T21:19:47Z] <ebernhardson@deploy1002> Synchronized php-1.39.0-wmf.19/extensions/CirrusSearch/includes/Job/ElasticaWrite.php: Backport: [[gerrit:811280|job queue: Squelch errors related to unwritable cloudelastic (T309648)]] (duration: 03m 43s)

Tue, Jul 5, 9:19 PM · Patch-For-Review, Discovery-Search (Current work)
Stashbot added a comment to T309648: Restore lost index in cloudelastic.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T20:24:54Z] <ebernhardson@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:811350|cirrus: Disable commonswiki writes to cloudelastic (T309648)]] (duration: 03m 23s)

Tue, Jul 5, 8:24 PM · Patch-For-Review, Discovery-Search (Current work)
Stashbot added a comment to T311015: Deploy QuickSurvey on Japanese Wikipedia.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T20:17:46Z] <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 66c973087b7736b22ce7edb5b830e50e31710e4a: QuickSurveys: Increase coverage of research-incentive survey (T311015) (duration: 03m 28s)

Tue, Jul 5, 8:17 PM · Performance-Team (Radar), Design-Research, MW-1.39-notes (1.39.0-wmf.18; 2022-06-27), Wikimedia-Site-requests, WikimediaMessages, Patch-For-Review
Stashbot added a comment to T307985: End GrowthExperiments welcome email campaign.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T20:12:12Z] <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: b1c217103753d886ab5b18b88f112ec26931bff2: GrowthExperiments: End mailing list campaign on eswiki (T307985) (duration: 03m 39s)

Tue, Jul 5, 8:12 PM · Growth-Team (Current Sprint)
Stashbot added a comment to T311686: Upgrade ganeti/codfw to Bullseye.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T15:00:52Z] <moritzm> draining ganeti2024 for eventual reimage T311686

Tue, Jul 5, 3:00 PM · Ganeti, Infrastructure-Foundations, SRE
Stashbot added a comment to T310684: Delete deprecated config from settings files.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T13:15:39Z] <urbanecm@deploy1002> Synchronized wmf-config/: 1287b969fc42aee6efae5ff1f1943394ba35e326: Drop deprecated feature flags (T310684) (duration: 03m 32s)

Tue, Jul 5, 1:16 PM · MW-1.39-notes (1.39.0-wmf.18; 2022-06-27), WMDE-TechWish-Sprint-2022-06-22, Patch-For-Review, WMDE-TechWish-Sprint-2022-06-08, WMDE-Templates-FocusArea
Stashbot added a comment to T310684: Delete deprecated config from settings files.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T13:08:27Z] <urbanecm@deploy1002> Synchronized wmf-config/CommonSettings.php: 891057f6ba555b2ece0424e3364d853eb20555da: Drop dependent feature flags (T310684) (duration: 03m 37s)

Tue, Jul 5, 1:08 PM · MW-1.39-notes (1.39.0-wmf.18; 2022-06-27), WMDE-TechWish-Sprint-2022-06-22, Patch-For-Review, WMDE-TechWish-Sprint-2022-06-08, WMDE-Templates-FocusArea
Stashbot added a comment to T311686: Upgrade ganeti/codfw to Bullseye.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T12:31:45Z] <moritzm> draining ganeti2023 for eventual reimage T311686

Tue, Jul 5, 12:31 PM · Ganeti, Infrastructure-Foundations, SRE
Stashbot added a comment to T311106: Investigate mariadb 10.6 performance regression during spikes/high load.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T12:29:41Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'T311106', diff saved to https://phabricator.wikimedia.org/P30859 and previous config saved to /var/cache/conftool/dbconfig/20220705-122941-ladsgroup.json

Tue, Jul 5, 12:29 PM · Sustainability (Incident Followup), SRE-OnFire, DBA
Stashbot added a comment to T311493: Productionize db2153.codfw.wmnet - db2174.codfw.wmnet.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T11:04:33Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Add db2158 to dbctl T311493', diff saved to https://phabricator.wikimedia.org/P30848 and previous config saved to /var/cache/conftool/dbconfig/20220705-110432-marostegui.json

Tue, Jul 5, 11:05 AM · Patch-For-Review, DBA
Stashbot added a comment to T311686: Upgrade ganeti/codfw to Bullseye.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T10:29:02Z] <moritzm> sudo gnt-cluster upgrade --to 3.0 for ganeti/codfw T311686

Tue, Jul 5, 10:29 AM · Ganeti, Infrastructure-Foundations, SRE
Stashbot added a comment to T308072: 1.39.0-wmf.19 deployment blockers.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T10:00:56Z] <jnuche@deploy1002> rebuilt and synchronized wikiversions files: group0 wikis to 1.39.0-wmf.19 refs T308072

Tue, Jul 5, 10:01 AM · Patch-For-Review, Release-Engineering-Team (Priority Backlog 📥), Release, Train Deployments
Stashbot added a comment to T308072: 1.39.0-wmf.19 deployment blockers.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T09:36:50Z] <jnuche@deploy1002> Finished scap: testwikis wikis to 1.39.0-wmf.19 refs T308072 (duration: 34m 21s)

Tue, Jul 5, 9:36 AM · Patch-For-Review, Release-Engineering-Team (Priority Backlog 📥), Release, Train Deployments
Stashbot added a comment to T308072: 1.39.0-wmf.19 deployment blockers.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T09:02:29Z] <jnuche@deploy1002> Started scap: testwikis wikis to 1.39.0-wmf.19 refs T308072

Tue, Jul 5, 9:02 AM · Patch-For-Review, Release-Engineering-Team (Priority Backlog 📥), Release, Train Deployments
Stashbot added a comment to T311390: Github's wikimedia/ores not mirroring to Gerrit's scoring/ores/ores.

Mentioned in SAL (#wikimedia-releng) [2022-07-05T08:49:31Z] <hauskatze> Diffusion rORES repository. Changed URI settings: enabled SSH push for mirroring; disabled HTTP | T311390

Tue, Jul 5, 8:49 AM · User-MarcoAurelio, Machine-Learning-Team (Active Tasks), Phabricator, Release-Engineering-Team
Stashbot added a comment to T311386: Install php 7.4 in production.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T08:30:41Z] <moritzm> uploaded 7.4.30-3+0~20220627.69+debian10~1.gbpf2b381+wmf1+buster3 to component/php74 (pulling php-common with the socket helper) T311386

Tue, Jul 5, 8:30 AM · Patch-For-Review, Performance-Team (Radar), serviceops
Stashbot added a comment to T300532: Migration of mentee overview to Vue.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T07:58:13Z] <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 89aef540e22aaded6c279d9d11c769507e497b6a: MentorDashboard: enable the Vue version of the dashboard in beta (T300532) (duration: 03m 18s)

Tue, Jul 5, 7:59 AM · Design-Systems-Team, Growth-Team (Current Sprint), Patch-For-Review, Vue.js, GrowthExperiments-MentorDashboard
Stashbot added a comment to T311946: Change the logo of Turkish Wikipedia to celebrate 500,000 articles.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T07:54:04Z] <urbanecm@deploy1002> Synchronized logos/config.yaml: c8c092a4133d119bf9aaece6f934ca7744ea6951: trwiki: Change old and new vector logos for 500k articles (T311946; 3/3) (duration: 03m 34s)

Tue, Jul 5, 7:54 AM · Logos, Wikimedia-Site-requests, Turkish-Sites
Stashbot added a comment to T311946: Change the logo of Turkish Wikipedia to celebrate 500,000 articles.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T07:50:29Z] <urbanecm@deploy1002> Synchronized wmf-config/: c8c092a4133d119bf9aaece6f934ca7744ea6951: trwiki: Change old and new vector logos for 500k articles (T311946; 2/3) (duration: 03m 36s)

Tue, Jul 5, 7:50 AM · Logos, Wikimedia-Site-requests, Turkish-Sites
Stashbot added a comment to T311946: Change the logo of Turkish Wikipedia to celebrate 500,000 articles.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T07:46:53Z] <urbanecm@deploy1002> Synchronized static/: c8c092a4133d119bf9aaece6f934ca7744ea6951: trwiki: Change old and new vector logos for 500k articles (T311946; 1/3) (duration: 03m 17s)

Tue, Jul 5, 7:47 AM · Logos, Wikimedia-Site-requests, Turkish-Sites
Stashbot added a comment to T311789: [wmf.18 - regression] Add image article card displays thumbnail image .

Mentioned in SAL (#wikimedia-operations) [2022-07-05T07:33:43Z] <urbanecm@deploy1002> Synchronized php-1.39.0-wmf.18/extensions/GrowthExperiments/includes/HomepageModules/SuggestedEdits.php: ce64780fbd78a414c6ab08fc374186ae4dd58bac: SuggestedEdits: Adjust thumbnailSource logic (T311789) (duration: 03m 32s)

Tue, Jul 5, 7:33 AM · Regression, Image-Suggestions, GrowthExperiments, Growth-Team
Stashbot added a comment to T311476: Unable to get list of more than 10k pages with recommendations.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T07:21:38Z] <urbanecm@deploy1002> Synchronized php-1.39.0-wmf.18/extensions/ImageSuggestions/maintenance/SendNotificationsForUnillustratedWatchedTitles.php: d5050b773992aa6100aa14cd328836ff336ef8c1: Retrieve pages-with-suggestion via Elastic scroll directly (T311476) (duration: 03m 32s)

Tue, Jul 5, 7:21 AM · Structured-Data-Backlog (Current Work), Image-Suggestions
Stashbot added a comment to T311944: Meta talkpages for Central Notice banners turned into banner overview pages from the admin interface.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T07:17:02Z] <urbanecm@deploy1002> Synchronized php-1.39.0-wmf.18/extensions/CentralNotice/includes/specials/CentralNotice.php: 414b7b8a14b451f9bd0fb0c36d44fe6a9310102e: Only add tabs to special pages (T311944) (duration: 03m 30s)

Tue, Jul 5, 7:17 AM · Fundraising Sprint Localhost Hospitality Studies, Fundraising-Backlog, MediaWiki-extensions-CentralNotice
Stashbot added a comment to T312012: Disable local upload for Chinese (Wikiversity|Wiktionary).

Mentioned in SAL (#wikimedia-operations) [2022-07-05T07:13:25Z] <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 14df0e25aabf21715b281a9dbb5893ae2ae7db9a: zh(wikiversity|wiktionary): Disable local upload (T312012) (duration: 03m 47s)

Tue, Jul 5, 7:14 AM · Chinese-Sites, Wikimedia-Site-requests
Stashbot added a comment to T311837: decommission db2073.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T07:00:20Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Decommission db2073 T311837', diff saved to https://phabricator.wikimedia.org/P30822 and previous config saved to /var/cache/conftool/dbconfig/20220705-070019-marostegui.json

Tue, Jul 5, 7:00 AM · SRE, ops-codfw, decommission-hardware
Stashbot added a comment to T298557: Fix mismatching field type of page.page_touched on wmf wikis.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T06:09:37Z] <marostegui> dbmaint s6@eqiad T298557

Tue, Jul 5, 6:09 AM · Schema-change-in-production, DBA
Stashbot added a comment to T311522: Switchover s6 master db1131 -> db1173.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T06:05:26Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1131 T311522', diff saved to https://phabricator.wikimedia.org/P30816 and previous config saved to /var/cache/conftool/dbconfig/20220705-060526-root.json

Tue, Jul 5, 6:05 AM · User-notice, DBA
Stashbot added a comment to T311522: Switchover s6 master db1131 -> db1173.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T06:01:39Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Promote db1173 to s6 primary and set section read-write T311522', diff saved to https://phabricator.wikimedia.org/P30815 and previous config saved to /var/cache/conftool/dbconfig/20220705-060139-root.json

Tue, Jul 5, 6:02 AM · User-notice, DBA
Stashbot added a comment to T311522: Switchover s6 master db1131 -> db1173.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T06:01:11Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Set s6 eqiad as read-only for maintenance - T311522', diff saved to https://phabricator.wikimedia.org/P30814 and previous config saved to /var/cache/conftool/dbconfig/20220705-060111-marostegui.json

Tue, Jul 5, 6:01 AM · User-notice, DBA
Stashbot added a comment to T311522: Switchover s6 master db1131 -> db1173.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T06:00:51Z] <marostegui> Starting s6 eqiad failover from db1131 to db1173 - T311522

Tue, Jul 5, 6:00 AM · User-notice, DBA
Stashbot added a comment to T311522: Switchover s6 master db1131 -> db1173.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T05:22:19Z] <marostegui@cumin1001> dbctl commit (dc=all): 'Set db1173 with weight 0 T311522', diff saved to https://phabricator.wikimedia.org/P30813 and previous config saved to /var/cache/conftool/dbconfig/20220705-052219-marostegui.json

Tue, Jul 5, 5:22 AM · User-notice, DBA
Stashbot added a comment to T311522: Switchover s6 master db1131 -> db1173.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T05:21:54Z] <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 23 hosts with reason: Primary switchover s6 T311522

Tue, Jul 5, 5:22 AM · User-notice, DBA
Stashbot added a comment to T311522: Switchover s6 master db1131 -> db1173.

Mentioned in SAL (#wikimedia-operations) [2022-07-05T05:21:26Z] <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on 23 hosts with reason: Primary switchover s6 T311522

Tue, Jul 5, 5:21 AM · User-notice, DBA

Mon, Jul 4

Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T19:29:56Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315 (T312027)', diff saved to https://phabricator.wikimedia.org/P30811 and previous config saved to /var/cache/conftool/dbconfig/20220704-192955-ladsgroup.json

Mon, Jul 4, 7:30 PM · DBA
Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T18:44:41Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315 (T312027)', diff saved to https://phabricator.wikimedia.org/P30808 and previous config saved to /var/cache/conftool/dbconfig/20220704-184440-ladsgroup.json

Mon, Jul 4, 6:44 PM · DBA
Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T18:42:34Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1096:3315 (T312027)', diff saved to https://phabricator.wikimedia.org/P30807 and previous config saved to /var/cache/conftool/dbconfig/20220704-184231-ladsgroup.json

Mon, Jul 4, 6:42 PM · DBA
Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T18:42:11Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1100 (T312027)', diff saved to https://phabricator.wikimedia.org/P30806 and previous config saved to /var/cache/conftool/dbconfig/20220704-184211-ladsgroup.json

Mon, Jul 4, 6:42 PM · DBA
Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T17:56:56Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1100 (T312027)', diff saved to https://phabricator.wikimedia.org/P30803 and previous config saved to /var/cache/conftool/dbconfig/20220704-175655-ladsgroup.json

Mon, Jul 4, 5:56 PM · DBA
Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T17:54:46Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1100 (T312027)', diff saved to https://phabricator.wikimedia.org/P30802 and previous config saved to /var/cache/conftool/dbconfig/20220704-175446-ladsgroup.json

Mon, Jul 4, 5:54 PM · DBA
Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T17:54:26Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1113:3315 (T312027)', diff saved to https://phabricator.wikimedia.org/P30801 and previous config saved to /var/cache/conftool/dbconfig/20220704-175425-ladsgroup.json

Mon, Jul 4, 5:54 PM · DBA
Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T17:09:11Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1113:3315 (T312027)', diff saved to https://phabricator.wikimedia.org/P30798 and previous config saved to /var/cache/conftool/dbconfig/20220704-170910-ladsgroup.json

Mon, Jul 4, 5:09 PM · DBA
Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T17:08:01Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1113:3315 (T312027)', diff saved to https://phabricator.wikimedia.org/P30797 and previous config saved to /var/cache/conftool/dbconfig/20220704-170800-ladsgroup.json

Mon, Jul 4, 5:08 PM · DBA
Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T17:07:41Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1110 (T312027)', diff saved to https://phabricator.wikimedia.org/P30796 and previous config saved to /var/cache/conftool/dbconfig/20220704-170740-ladsgroup.json

Mon, Jul 4, 5:07 PM · DBA
Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T16:22:26Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1110 (T312027)', diff saved to https://phabricator.wikimedia.org/P30792 and previous config saved to /var/cache/conftool/dbconfig/20220704-162225-ladsgroup.json

Mon, Jul 4, 4:22 PM · DBA
Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T16:20:15Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1110 (T312027)', diff saved to https://phabricator.wikimedia.org/P30791 and previous config saved to /var/cache/conftool/dbconfig/20220704-162015-ladsgroup.json

Mon, Jul 4, 4:20 PM · DBA
Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T16:19:44Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T312027)', diff saved to https://phabricator.wikimedia.org/P30790 and previous config saved to /var/cache/conftool/dbconfig/20220704-161944-ladsgroup.json

Mon, Jul 4, 4:19 PM · DBA
Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T15:34:29Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T312027)', diff saved to https://phabricator.wikimedia.org/P30784 and previous config saved to /var/cache/conftool/dbconfig/20220704-153428-ladsgroup.json

Mon, Jul 4, 3:34 PM · DBA
Stashbot added a comment to T312027: Fix enwikivoyage drifts on pagelinks.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T15:32:19Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1144:3315 (T312027)', diff saved to https://phabricator.wikimedia.org/P30782 and previous config saved to /var/cache/conftool/dbconfig/20220704-153218-ladsgroup.json

Mon, Jul 4, 3:32 PM · DBA
Stashbot added a comment to T305300: Add lu_attachment_method column to localuser table.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T15:29:31Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1161 (T305300)', diff saved to https://phabricator.wikimedia.org/P30781 and previous config saved to /var/cache/conftool/dbconfig/20220704-152931-ladsgroup.json

Mon, Jul 4, 3:29 PM · User-Ladsgroup, DBA, Schema-change-in-production, MediaWiki-extensions-CentralAuth
Stashbot added a comment to T307648: Audit database usage of GlobalBlocking extension.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T14:27:16Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:810932|Exempt WMCS ranges from globalblocking everywhere (T307648)]] (duration: 03m 26s)

Mon, Jul 4, 2:27 PM · Patch-For-Review, MW-1.39-notes (1.39.0-wmf.14; 2022-05-30), GlobalBlocking, Sustainability (Incident Followup), DBA
Stashbot added a comment to T311628: Create Swift account for readonly access to ML models.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T14:19:43Z] <elukey> roll restart of thanos-fe's proxy to pick up a new account - T311628

Mon, Jul 4, 2:19 PM · Machine-Learning-Team (Active Tasks), SRE-swift-storage, Lift-Wing
Stashbot added a comment to T307648: Audit database usage of GlobalBlocking extension.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T14:10:42Z] <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:810055|Set GlobalBlockingAllowedRanges for testwiki (T307648)]] (duration: 03m 39s)

Mon, Jul 4, 2:10 PM · Patch-For-Review, MW-1.39-notes (1.39.0-wmf.14; 2022-05-30), GlobalBlocking, Sustainability (Incident Followup), DBA
Stashbot added a comment to T283017: Create a dashboard for database backups monitoring/reporting.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T12:38:22Z] <jynus> running alter table on dbbackups db T283017

Mon, Jul 4, 12:38 PM · dbbackups-dashboard, Patch-For-Review, Goal, database-backups, Data-Persistence-Backup
Stashbot added a comment to T307648: Audit database usage of GlobalBlocking extension.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T11:55:22Z] <ladsgroup@deploy1002> Synchronized php-1.39.0-wmf.18/extensions/GlobalBlocking/includes/GlobalBlocking.php: Backport: [[gerrit:810518|Add statsd metric collection on db calls (T307648)]] (duration: 03m 26s)

Mon, Jul 4, 11:55 AM · Patch-For-Review, MW-1.39-notes (1.39.0-wmf.14; 2022-05-30), GlobalBlocking, Sustainability (Incident Followup), DBA
Stashbot added a comment to T311916: "Add an image" structured edits add a blank line instead of an image.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T11:50:03Z] <ladsgroup@deploy1002> Synchronized php-1.39.0-wmf.18/extensions/GrowthExperiments/modules/ext.growthExperiments.StructuredTask/addimage/AddImageArticleTarget.js: Backport: [[gerrit:810509|AddImageArticleTarget: Update to new mediaClass/mediaTag format (T311916)]] (duration: 03m 33s)

Mon, Jul 4, 11:50 AM · Patch-For-Review, Regression, GrowthExperiments-NewcomerTasks, Image-Suggestions, Growth-Team (Current Sprint)
Stashbot added a comment to T311493: Productionize db2153.codfw.wmnet - db2174.codfw.wmnet.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T11:36:41Z] <marostegui@cumin2002> dbctl commit (dc=all): 'Add db2156 to s3 T311493', diff saved to https://phabricator.wikimedia.org/P30774 and previous config saved to /var/cache/conftool/dbconfig/20220704-113640-marostegui.json

Mon, Jul 4, 11:37 AM · Patch-For-Review, DBA
Stashbot added a comment to T311360: RecentChanges timing out.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T10:54:55Z] <ladsgroup@deploy1002> Synchronized php-1.39.0-wmf.18/includes: Backport: [[gerrit:810139|Revert "Revert "RecentChange: Straight join to actor table when needed"" (T311360)]] (duration: 03m 49s)

Mon, Jul 4, 10:55 AM · Upstream, mariadb-optimizer-bug, Slow-DB-Query, Performance-Team, DBA, Platform Engineering, Vuln-DoS, Wikimedia-production-error, Growth-Team, MediaWiki-Recent-changes
Stashbot added a comment to T311493: Productionize db2153.codfw.wmnet - db2174.codfw.wmnet.

Mentioned in SAL (#wikimedia-operations) [2022-07-04T08:24:07Z] <marostegui@cumin2002> dbctl commit (dc=all): 'Add db2157 to s5 T311493', diff saved to https://phabricator.wikimedia.org/P30758 and previous config saved to /var/cache/conftool/dbconfig/20220704-082406-marostegui.json

Mon, Jul 4, 8:24 AM · Patch-For-Review, DBA

Fri, Jul 1

Stashbot added a comment to T309311: Make user_editcount unsigned in production.

Mentioned in SAL (#wikimedia-operations) [2022-07-01T23:55:24Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1118 (T309311)', diff saved to https://phabricator.wikimedia.org/P30753 and previous config saved to /var/cache/conftool/dbconfig/20220701-235524-ladsgroup.json

Fri, Jul 1, 11:55 PM · Schema-change-in-production, DBA
Stashbot added a comment to T309311: Make user_editcount unsigned in production.

Mentioned in SAL (#wikimedia-operations) [2022-07-01T23:10:09Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1118 (T309311)', diff saved to https://phabricator.wikimedia.org/P30750 and previous config saved to /var/cache/conftool/dbconfig/20220701-231009-ladsgroup.json

Fri, Jul 1, 11:10 PM · Schema-change-in-production, DBA
Stashbot added a comment to T309311: Make user_editcount unsigned in production.

Mentioned in SAL (#wikimedia-operations) [2022-07-01T22:14:39Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1118 (T309311)', diff saved to https://phabricator.wikimedia.org/P30749 and previous config saved to /var/cache/conftool/dbconfig/20220701-221438-ladsgroup.json

Fri, Jul 1, 10:14 PM · Schema-change-in-production, DBA
Stashbot added a comment to T309311: Make user_editcount unsigned in production.

Mentioned in SAL (#wikimedia-operations) [2022-07-01T22:14:19Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1119 (T309311)', diff saved to https://phabricator.wikimedia.org/P30748 and previous config saved to /var/cache/conftool/dbconfig/20220701-221418-ladsgroup.json

Fri, Jul 1, 10:14 PM · Schema-change-in-production, DBA
Stashbot added a comment to T311890: restbase2018 down .

Mentioned in SAL (#wikimedia-operations) [2022-07-01T22:12:38Z] <mutante> restbase2018 - attempting power cycle via mgmt - /admin1-> racadm serveraction powercycle (T311890)

Fri, Jul 1, 10:12 PM · ops-codfw, SRE
Stashbot added a comment to T247653: replace doc1001.eqiad.wmnet with a buster VM and create the codfw equivalent.

Mentioned in SAL (#wikimedia-operations) [2022-07-01T21:48:45Z] <mutante> https://doc.wikimedia.org switched to doc1002 backend on buster T247653

Fri, Jul 1, 9:48 PM · serviceops-collab, Release-Engineering-Team (Seen), Patch-For-Review, Continuous-Integration-Infrastructure, serviceops, SRE
Stashbot added a comment to T309311: Make user_editcount unsigned in production.

Mentioned in SAL (#wikimedia-operations) [2022-07-01T21:29:04Z] <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1119 (T309311)', diff saved to https://phabricator.wikimedia.org/P30745 and previous config saved to /var/cache/conftool/dbconfig/20220701-212903-ladsgroup.json

Fri, Jul 1, 9:29 PM · Schema-change-in-production, DBA