Page MenuHomePhabricator

jijiki (effie mouzeli)
is an animal

Projects (11)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Aug 14 2018, 10:50 AM (147 w, 6 d)
Availability
Available
IRC Nick
effie
LDAP User
Effie Mouzeli
MediaWiki User
EMouzeli (WMF) [ Global Accounts ]

Recent Activity

Fri, Jun 11

jijiki updated the task description for T283159: Deploy tegola-vector-tiles to kubernetes.
Fri, Jun 11, 2:36 PM · Patch-For-Review, User-jijiki, serviceops, Maps
jijiki committed rLPRI8e343c1cbe84: Add tokens and users for tegola-vector-tiles (authored by jijiki).
Add tokens and users for tegola-vector-tiles
Fri, Jun 11, 12:48 PM

Wed, Jun 9

jijiki committed rLPRIbae946428271: Add redis password for mw:nutcracker:redis_password (authored by jijiki).
Add redis password for mw:nutcracker:redis_password
Wed, Jun 9, 9:16 AM
jijiki renamed T284628: Kubernetes timeing out before pulling the mediawiki-multiversion image from Kubernetes timeing out befor pulling the mediawiki-multiversion image to Kubernetes timeing out before pulling the mediawiki-multiversion image .
Wed, Jun 9, 8:15 AM · Kubernetes, serviceops, MW-on-K8s
jijiki added a subtask for T264209: Run stress tests on docker images infrastructure: T284628: Kubernetes timeing out before pulling the mediawiki-multiversion image .
Wed, Jun 9, 7:45 AM · Patch-For-Review, serviceops, Release Pipeline, MW-on-K8s
jijiki added a parent task for T284628: Kubernetes timeing out before pulling the mediawiki-multiversion image : T264209: Run stress tests on docker images infrastructure.
Wed, Jun 9, 7:44 AM · Kubernetes, serviceops, MW-on-K8s
jijiki created T284628: Kubernetes timeing out before pulling the mediawiki-multiversion image .
Wed, Jun 9, 7:42 AM · Kubernetes, serviceops, MW-on-K8s

Mon, Jun 7

jijiki added a project to T284454: Refactor memcached modules : User-jijiki.
Mon, Jun 7, 2:09 PM · User-jbond, User-jijiki, SRE, serviceops
jijiki created T284454: Refactor memcached modules .
Mon, Jun 7, 2:09 PM · User-jbond, User-jijiki, SRE, serviceops
jijiki added a comment to T284421: Enable TLS termination on the mwdebug deployment. fix the service definition in the chart.

I have enabled TLS on staging for now, which will use some default certs

Mon, Jun 7, 10:33 AM · MW-on-K8s, serviceops, SRE
jijiki closed T284301: Outdated link in "MediaWiki exceptions and fatals" alert as Resolved.
Mon, Jun 7, 5:41 AM · serviceops, observability

Tue, Jun 1

jijiki moved T283159: Deploy tegola-vector-tiles to kubernetes from Inbox 🐅 to Next up 🥌 on the User-jijiki board.
Tue, Jun 1, 10:08 AM · Patch-For-Review, User-jijiki, serviceops, Maps

Thu, May 27

jijiki closed T283744: codfw appserver latency alerts flapping as Resolved.

Unless @JMeybohm was not the culprit, we can mark this as resolved

Thu, May 27, 8:04 PM · serviceops

Wed, May 26

jijiki added a comment to T283744: codfw appserver latency alerts flapping.

Ι think there is a correlation between the latency spikes and TCP retransmits

Wed, May 26, 9:34 PM · serviceops
jijiki added a comment to T271967: Enable TLS on memcached for cross-dc replication.

i wonder if we have considered just having the TLS port every where accept localhost?

regardless i guess we need a transition config but still curious if this if that is the long trm goal or not?

Wed, May 26, 4:55 PM · Patch-For-Review, Performance-Team (Radar), User-jijiki, SRE, serviceops
jijiki placed T221081: Replace nutcracker with mcrouter on thumbor* up for grabs.
Wed, May 26, 12:46 PM · User-jijiki, serviceops, SRE, Thumbor

Tue, May 25

jijiki updated the task description for T271967: Enable TLS on memcached for cross-dc replication.
Tue, May 25, 6:23 PM · Patch-For-Review, Performance-Team (Radar), User-jijiki, SRE, serviceops
jijiki closed T235755: Increased POST latency for MW app servers (Oct 2019) as Resolved.

No!

Tue, May 25, 6:15 AM · Performance-Team (Radar), serviceops, SRE

Mon, May 24

jijiki renamed T283159: Deploy tegola-vector-tiles to kubernetes from Deploy maps-vector-server to kubernetes to Deploy tegola-vector-tiles to kubernetes.
Mon, May 24, 4:42 PM · Patch-For-Review, User-jijiki, serviceops, Maps
jijiki renamed T274390: New Service Request tegola-vector-tiles from New Service Request maps-vector-server to New Service Request tegola-vector-tiles.
Mon, May 24, 4:41 PM · serviceops, Maps, Product-Infrastructure-Team-Backlog, Service-deployment-requests, Services, SRE
jijiki added a parent task for T283159: Deploy tegola-vector-tiles to kubernetes: T274390: New Service Request tegola-vector-tiles.
Mon, May 24, 4:40 PM · Patch-For-Review, User-jijiki, serviceops, Maps
jijiki added a subtask for T274390: New Service Request tegola-vector-tiles: T283159: Deploy tegola-vector-tiles to kubernetes.
Mon, May 24, 4:40 PM · serviceops, Maps, Product-Infrastructure-Team-Backlog, Service-deployment-requests, Services, SRE

Sat, May 22

JoKalliauer awarded T212941: Investigate systemd hardening to replace Firejail for Thumbor a Like token.
Sat, May 22, 6:13 PM · Patch-For-Review, User-jijiki, serviceops, SRE, Thumbor

Thu, May 20

jijiki committed rLPRI9a0c30a3f2e6: Add kubernetes mwdebug user (authored by jijiki).
Add kubernetes mwdebug user
Thu, May 20, 4:13 PM
jijiki updated the task description for T283056: Create a mwdebug deployment for mediawiki on kubernetes.
Thu, May 20, 4:12 PM · MW-on-K8s, serviceops, SRE
jijiki committed rLPRI4c4b84f0929d: Add tokens for mwdebug service (authored by jijiki).
Add tokens for mwdebug service
Thu, May 20, 6:57 AM

Wed, May 19

jijiki created T283159: Deploy tegola-vector-tiles to kubernetes.
Wed, May 19, 1:42 PM · Patch-For-Review, User-jijiki, serviceops, Maps
jijiki closed T242861: Clarify multi-service instance concepts in helm charts and enable canary releases as Resolved.

Continued in T282148

Wed, May 19, 12:19 PM · Analytics, serviceops
jijiki added a project to T283049: Swift account to store pre-rendered vector-tiles : User-jijiki.
Wed, May 19, 10:58 AM · User-jijiki, serviceops, Maps, SRE-swift-storage
jijiki removed a project from T242861: Clarify multi-service instance concepts in helm charts and enable canary releases: User-jijiki.
Wed, May 19, 10:57 AM · Analytics, serviceops
jijiki added a comment to T242861: Clarify multi-service instance concepts in helm charts and enable canary releases.

@akosiaris @Ottomata should we resolve this in favour of T282148 ?

Wed, May 19, 10:57 AM · Analytics, serviceops
jijiki moved T273950: Modernise memcached systemd unit / sync to current buster setup from Next up 🥌 to St on the User-jijiki board.
Wed, May 19, 10:56 AM · serviceops, User-jijiki, SRE
jijiki removed a project from T245757: Upgrade MediaWiki clusters to Debian Buster (debian 10): User-jijiki.
Wed, May 19, 10:54 AM · Release-Engineering-Team (Radar), SRE, serviceops
jijiki moved T271967: Enable TLS on memcached for cross-dc replication from In Progress 🏋️‍♀️ to Next up 🥌 on the User-jijiki board.
Wed, May 19, 10:54 AM · Patch-For-Review, Performance-Team (Radar), User-jijiki, SRE, serviceops
jijiki moved T270315: Upgrade memcached to version 1.6.x from In Progress 🏋️‍♀️ to Q4 2020 on the User-jijiki board.
Wed, May 19, 10:54 AM · Patch-For-Review, User-jijiki, SRE, serviceops

Tue, May 18

jijiki renamed T274390: New Service Request tegola-vector-tiles from New Service Request tegola to New Service Request maps-vector-server.
Tue, May 18, 5:32 PM · serviceops, Maps, Product-Infrastructure-Team-Backlog, Service-deployment-requests, Services, SRE
jijiki updated the task description for T283056: Create a mwdebug deployment for mediawiki on kubernetes.
Tue, May 18, 5:21 PM · MW-on-K8s, serviceops, SRE
jijiki removed a project from T274390: New Service Request tegola-vector-tiles: Kubernetes.
Tue, May 18, 4:17 PM · serviceops, Maps, Product-Infrastructure-Team-Backlog, Service-deployment-requests, Services, SRE
jijiki added projects to T274390: New Service Request tegola-vector-tiles: serviceops, Kubernetes.
Tue, May 18, 4:15 PM · serviceops, Maps, Product-Infrastructure-Team-Backlog, Service-deployment-requests, Services, SRE
jijiki added a subtask for T274390: New Service Request tegola-vector-tiles: T283049: Swift account to store pre-rendered vector-tiles .
Tue, May 18, 6:14 AM · serviceops, Maps, Product-Infrastructure-Team-Backlog, Service-deployment-requests, Services, SRE
jijiki added a parent task for T283049: Swift account to store pre-rendered vector-tiles : T274390: New Service Request tegola-vector-tiles.
Tue, May 18, 6:13 AM · User-jijiki, serviceops, Maps, SRE-swift-storage
jijiki created T283049: Swift account to store pre-rendered vector-tiles .
Tue, May 18, 6:13 AM · User-jijiki, serviceops, Maps, SRE-swift-storage

May 7 2021

Ottomata awarded T282148: Support Canary releases on Kubernetes a Insectivore token.
May 7 2021, 4:51 PM · serviceops
jijiki updated the task description for T282148: Support Canary releases on Kubernetes .
May 7 2021, 3:30 PM · serviceops
jijiki added a parent task for T282148: Support Canary releases on Kubernetes : T210143: Canaries canaries canaries.
May 7 2021, 3:12 PM · serviceops
jijiki added a subtask for T210143: Canaries canaries canaries: T282148: Support Canary releases on Kubernetes .
May 7 2021, 3:12 PM · User-brennen, User-WDoran, serviceops, SRE
jijiki added projects to T282148: Support Canary releases on Kubernetes : serviceops, SRE.
May 7 2021, 3:10 PM · serviceops
jijiki updated the task description for T282148: Support Canary releases on Kubernetes .
May 7 2021, 3:07 PM · serviceops

May 6 2021

jijiki created T282148: Support Canary releases on Kubernetes .
May 6 2021, 2:46 PM · serviceops
jijiki closed T279695: Deploy Scap version 3.17.1-1 as Resolved.

@dancy it appears that rolling out scap coincided with the greek easter this time , sorry!

May 6 2021, 5:37 AM · Release-Engineering-Team (Radar), serviceops, Scap
jijiki updated the task description for T281135: codfw: Relocate servers in 10G racks .
May 6 2021, 4:44 AM · Data-Persistence (Consultation), serviceops, SRE, ops-codfw

May 5 2021

jijiki added a project to T281135: codfw: Relocate servers in 10G racks : serviceops.
May 5 2021, 9:13 AM · Data-Persistence (Consultation), serviceops, SRE, ops-codfw

May 1 2021

jijiki added a parent task for T276415: mc1027.eqiad.wmnet is down, not powering back up: T281618: decommission mc1027.eqiad.wmnet.
May 1 2021, 3:09 AM · SRE, ops-eqiad
jijiki added a subtask for T281618: decommission mc1027.eqiad.wmnet: T276415: mc1027.eqiad.wmnet is down, not powering back up.
May 1 2021, 3:09 AM · serviceops, decommission-hardware
jijiki created T281618: decommission mc1027.eqiad.wmnet.
May 1 2021, 3:08 AM · serviceops, decommission-hardware

Apr 29 2021

jijiki updated subscribers of T280579: New service request: WDQS Flink based Streaming Updater.
Apr 29 2021, 9:51 AM · Discovery-Search (Current work), wdwb-tech, SRE, Services, Wikidata, Service-deployment-requests, Wikidata-Query-Service

Apr 27 2021

jijiki added a comment to T279695: Deploy Scap version 3.17.1-1.

@dancy thank you very much, everything worked well!
I have installed scap 3.17.1-1 on our api canaries, I will proceed with a full rollout this week.

Apr 27 2021, 4:27 PM · Release-Engineering-Team (Radar), serviceops, Scap
jijiki claimed T281261: Update grafana link for mediawiki-error-rate-$cluster in icinga check.
Apr 27 2021, 3:57 PM · serviceops, SRE

Apr 26 2021

jijiki added a comment to T279100: Have some dedicated jobrunners that aren't active videoscalers.

So, the crux of the issue is at those 2 functions below

def pool(self, pooled):
    """Pool a list of services"""
    try:
        for obj in pooled:
            obj.update({"pooled": "yes"})
        self._verify_status(True)
        return True 
    except (BackendError, PoolStatusError) as e:
        logger.error("Error depooling the servers: %s", e)
        return False

def _verify_status(self, want_pooled):
    if want_pooled:
        desired_status = "enabled/up/pooled"
    else:
        desired_status = "disabled/*/not pooled"
    for baseurl in self.lvs_uris:
        parsed = parse.urlparse(baseurl)
        url = "{baseurl}/{fqdn}".format(baseurl=baseurl, fqdn=self.fqdn)
        # This will raise a PoolStatusError
        logger.debug("Now verifying %s", url) 
        self._fetch_retry(url, want_pooled, parsed, desired_status)

The pool() function will get a list of services to pool, in the specific case the jeena raised, it will only pool the service that was pooled already before the script started (that is the jobrunner service) and leave the videoscaler service depooled (correctly). However it calls the _verify_status() function that has no such knowledge of previous state and relies only on the lvs uris passed on the command line.

Let me see if I can fix that easily.

Apr 26 2021, 11:40 AM · Patch-For-Review, WMF-JobQueue, Sustainability (Incident Followup), SRE, serviceops
jijiki added a project to T281107: ms-be1062 fell off the network, causing swift timeouts: Wikimedia-Incident.
Apr 26 2021, 9:52 AM · Wikimedia-Incident, SRE, SRE-swift-storage
jijiki closed T254845: Build python-thumbor-wikimedia 2.9 Debian package and upload to apt.wikimedia.org as Resolved.

It has been deployed to production, I forgot to close that task, sorry!

Apr 26 2021, 8:44 AM · User-jijiki, Performance-Team (Radar), serviceops, Thumbor, SRE

Apr 24 2021

JoKalliauer awarded T216815: Upgrade Thumbor to Buster a Like token.
Apr 24 2021, 5:40 PM · User-jijiki, serviceops, SRE, Thumbor
jijiki closed T261009: Reproduce opcache corruptions in production as Resolved.

The reasons have been identified in https://phabricator.wikimedia.org/T253673#6569013

Apr 24 2021, 5:38 AM · User-jijiki, serviceops, SRE
jijiki closed T261009: Reproduce opcache corruptions in production , a subtask of T253673: Avoid php-opcache corruption in WMF production, as Resolved.
Apr 24 2021, 5:38 AM · Performance-Team (Radar), User-jijiki, Patch-For-Review, Sustainability (Incident Followup), serviceops

Apr 23 2021

jijiki reopened T270315: Upgrade memcached to version 1.6.x as "Open".
Apr 23 2021, 3:22 PM · Patch-For-Review, User-jijiki, SRE, serviceops
jijiki updated the task description for T270315: Upgrade memcached to version 1.6.x.
Apr 23 2021, 3:21 PM · Patch-For-Review, User-jijiki, SRE, serviceops
jijiki reopened T270315: Upgrade memcached to version 1.6.x, a subtask of T244852: Upgrade and improve our application object caching service (memcached), as Open.
Apr 23 2021, 3:21 PM · Patch-For-Review, SRE, serviceops
jijiki reopened T270315: Upgrade memcached to version 1.6.x, a subtask of T264604: Enable "/*/mw-with-onhost-tier/" route for MediaWiki where safe, as Open.
Apr 23 2021, 3:21 PM · MW-1.36-notes, MW-1.37-notes (1.37.0-wmf.1; 2021-04-13), Patch-For-Review, User-jijiki, SRE, serviceops, Performance-Team
jijiki reopened T270315: Upgrade memcached to version 1.6.x, a subtask of T271967: Enable TLS on memcached for cross-dc replication, as Open.
Apr 23 2021, 3:21 PM · Patch-For-Review, Performance-Team (Radar), User-jijiki, SRE, serviceops
jijiki moved T266055: Update Scap to perform rolling restart for all MW deploy from Inbox 🐅 to In Progress 🏋️‍♀️ on the User-jijiki board.
Apr 23 2021, 2:52 PM · Release-Engineering-Team (Radar), User-jijiki, Patch-For-Review, Scap

Apr 21 2021

jijiki triaged T280843: Find 8 machines (4 eqiad + 4 codfw) for Thumbor as Medium priority.
Apr 21 2021, 7:40 PM · serviceops, SRE
jijiki added a project to T275133: Limit query parallelism from Flink based WDQS updater to Wikidata: User-jijiki.
Apr 21 2021, 2:38 PM · User-jijiki, wdwb-tech, Discovery-Search (Current work), Wikidata-Query-Service, serviceops, Wikidata
jijiki added projects to T280767: Maps 2.0 roll-out plan: serviceops, User-jijiki.
Apr 21 2021, 1:11 PM · User-jijiki, serviceops, Maps, Product-Infrastructure-Team-Backlog
jijiki added a comment to T279695: Deploy Scap version 3.17.1-1.

@LarsWirzenius as discussed in T265501, using docker to build scap, is not an option. In T277793#6957246 I wrote that "we will be happy with a step by step documented process of how to build the debian package on our build host". Please update the "Building" section in https://wikitech.wikimedia.org/wiki/Scap3#Production_Upgrade, and I will be happy to build and roll it out.

Apr 21 2021, 7:06 AM · Release-Engineering-Team (Radar), serviceops, Scap

Apr 20 2021

jijiki closed T280642: conftool unable to announce changes to icinga.wikimedia.org:9200 as Invalid.
Apr 20 2021, 9:55 AM · serviceops, SRE, observability
jijiki triaged T280642: conftool unable to announce changes to icinga.wikimedia.org:9200 as High priority.
Apr 20 2021, 9:49 AM · serviceops, SRE, observability
jijiki created T280642: conftool unable to announce changes to icinga.wikimedia.org:9200.
Apr 20 2021, 9:49 AM · serviceops, SRE, observability

Apr 19 2021

Krinkle awarded T280582: Shrink redis_sessions cluster a Orange Medal token.
Apr 19 2021, 7:35 PM · Performance-Team (Radar), User-jijiki, SRE, serviceops
jijiki added a comment to T280582: Shrink redis_sessions cluster.

Note that deployment of T113916 was halted because the Redis capacity was actually considered too small. (That task is about migrating module dep store from a MW core DB table written to during GET requests, to instead use Main Stash)

I do believe however that after MainStash DB (x2, T113916) is fully online and configured as backend for MainStash in MW, that indeed this cluster can be downsized, even much further to only 3 or 4 servers perhaps? ChronologyProtector would likely be its only consumer, perhaps with one or two other consumers besides it that store small and short-lived dc-local values at an every lower frequency.

Apr 19 2021, 7:15 PM · Performance-Team (Radar), User-jijiki, SRE, serviceops
jijiki renamed T277183: Phase out nutcracker for connecting to redis from Get rid of nutcracker for connecting to redis to Phase out nutcracker for connecting to redis.
Apr 19 2021, 7:09 PM · Performance-Team (Radar), SRE, serviceops
jijiki added a subtask for T267581: Phasing out "redis_sessions" MediaWiki cluster and away from the memcached cluster: T280586: Move "redis_sessions" to "redis_misc" cluster.
Apr 19 2021, 7:08 PM · Performance-Team (Radar), Platform Engineering, serviceops, SRE
jijiki added a parent task for T280586: Move "redis_sessions" to "redis_misc" cluster: T267581: Phasing out "redis_sessions" MediaWiki cluster and away from the memcached cluster.
Apr 19 2021, 7:08 PM · serviceops, SRE
jijiki triaged T280586: Move "redis_sessions" to "redis_misc" cluster as Medium priority.
Apr 19 2021, 7:07 PM · serviceops, SRE
jijiki created T280586: Move "redis_sessions" to "redis_misc" cluster.
Apr 19 2021, 7:06 PM · serviceops, SRE
jijiki renamed T267581: Phasing out "redis_sessions" MediaWiki cluster and away from the memcached cluster from Phasing out "redis_sessions" MediaWiki cluster from the memcached cluster to Phasing out "redis_sessions" MediaWiki cluster and away from the memcached cluster.
Apr 19 2021, 6:59 PM · Performance-Team (Radar), Platform Engineering, serviceops, SRE
jijiki renamed T267581: Phasing out "redis_sessions" MediaWiki cluster and away from the memcached cluster from Phasing out "redis_sessions" MediaWiki cluster to Phasing out "redis_sessions" MediaWiki cluster from the memcached cluster.
Apr 19 2021, 6:53 PM · Performance-Team (Radar), Platform Engineering, serviceops, SRE
jijiki added a subtask for T267581: Phasing out "redis_sessions" MediaWiki cluster and away from the memcached cluster: T280582: Shrink redis_sessions cluster.
Apr 19 2021, 6:52 PM · Performance-Team (Radar), Platform Engineering, serviceops, SRE
jijiki added a parent task for T280582: Shrink redis_sessions cluster: T267581: Phasing out "redis_sessions" MediaWiki cluster and away from the memcached cluster.
Apr 19 2021, 6:52 PM · Performance-Team (Radar), User-jijiki, SRE, serviceops
jijiki created T280582: Shrink redis_sessions cluster.
Apr 19 2021, 6:52 PM · Performance-Team (Radar), User-jijiki, SRE, serviceops
jijiki merged T243520: Decommission the "session redis" cluster into T267581: Phasing out "redis_sessions" MediaWiki cluster and away from the memcached cluster.
Apr 19 2021, 6:42 PM · Performance-Team (Radar), Platform Engineering, serviceops, SRE
jijiki added a subtask for T267581: Phasing out "redis_sessions" MediaWiki cluster and away from the memcached cluster: T277183: Phase out nutcracker for connecting to redis.
Apr 19 2021, 6:42 PM · Performance-Team (Radar), Platform Engineering, serviceops, SRE
jijiki added a parent task for T277183: Phase out nutcracker for connecting to redis: T267581: Phasing out "redis_sessions" MediaWiki cluster and away from the memcached cluster.
Apr 19 2021, 6:42 PM · Performance-Team (Radar), SRE, serviceops
jijiki merged task T243520: Decommission the "session redis" cluster into T267581: Phasing out "redis_sessions" MediaWiki cluster and away from the memcached cluster.
Apr 19 2021, 6:42 PM · Performance-Team, Platform Engineering, Goal, SRE

Apr 15 2021

Dzahn awarded T268524: Upgrade Parsoid servers to buster a Party Time token.
Apr 15 2021, 3:47 PM · Patch-For-Review, Parsoid (Tracking), serviceops, SRE
jijiki closed T268524: Upgrade Parsoid servers to buster as Resolved.
Apr 15 2021, 3:43 PM · Patch-For-Review, Parsoid (Tracking), serviceops, SRE
jijiki closed T268524: Upgrade Parsoid servers to buster , a subtask of T245757: Upgrade MediaWiki clusters to Debian Buster (debian 10), as Resolved.
Apr 15 2021, 3:43 PM · Release-Engineering-Team (Radar), SRE, serviceops
jijiki updated the task description for T268524: Upgrade Parsoid servers to buster .
Apr 15 2021, 3:42 PM · Patch-For-Review, Parsoid (Tracking), serviceops, SRE

Apr 14 2021

jijiki closed T273115: Migrate onhost memcached to use a unix socket as Resolved.
Apr 14 2021, 11:11 AM · Patch-For-Review, serviceops, SRE
jijiki closed T273115: Migrate onhost memcached to use a unix socket, a subtask of T264604: Enable "/*/mw-with-onhost-tier/" route for MediaWiki where safe, as Resolved.
Apr 14 2021, 11:11 AM · MW-1.36-notes, MW-1.37-notes (1.37.0-wmf.1; 2021-04-13), Patch-For-Review, User-jijiki, SRE, serviceops, Performance-Team
jijiki added a comment to T279100: Have some dedicated jobrunners that aren't active videoscalers.

I 've gone a bit overboard and created https://gerrit.wikimedia.org/r/679258 that uses YAML map merges to implement the patches above also in the structures. That will probably help avoid some unfortunate typo or mistake where the actual state in conftool differs from what we 've documented. What do you think?

Apr 14 2021, 8:39 AM · Patch-For-Review, WMF-JobQueue, Sustainability (Incident Followup), SRE, serviceops