Page MenuHomePhabricator

taavi (Taavi Väänänen)
User

Projects (26)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Saturday

  • Clear sailing ahead.

User Details

User Since
Feb 24 2019, 3:58 PM (188 w, 4 d)
Availability
Available
IRC Nick
taavi
LDAP User
Majavah
MediaWiki User
Taavi [ Global Accounts ]

Recent Activity

Today

taavi placed T241285: Deployment-prep should host its own statsd/graphite server up for grabs.
Thu, Oct 6, 5:42 AM · Patch-For-Review, Beta-Cluster-Infrastructure

Yesterday

taavi added a parent task for T305828: upgrade cloud-vps openstack to Openstack version 'Yoga': Unknown Object (Task).
Wed, Oct 5, 7:32 AM · cloud-services-team (Kanban), Cloud-VPS
taavi removed a subtask for T305828: upgrade cloud-vps openstack to Openstack version 'Yoga': Unknown Object (Task).
Wed, Oct 5, 7:32 AM · cloud-services-team (Kanban), Cloud-VPS
taavi added a subtask for T305828: upgrade cloud-vps openstack to Openstack version 'Yoga': Unknown Object (Task).
Wed, Oct 5, 7:32 AM · cloud-services-team (Kanban), Cloud-VPS
taavi closed T319367: New upstream release for covers-validator as Resolved.
Wed, Oct 5, 5:49 AM · Patch-For-Review, LibUp

Tue, Oct 4

taavi closed T267194: CloudVPS: enable TLS in openstack API endpoints, a subtask of T319312: Open Openstack APIs to the public internet, as Resolved.
Tue, Oct 4, 7:26 PM · Patch-For-Review, cloud-services-team (Kanban)
taavi closed T267194: CloudVPS: enable TLS in openstack API endpoints as Resolved.
Tue, Oct 4, 7:26 PM · cloud-services-team (Kanban), Patch-For-Review, Cloud-VPS
taavi removed a subtask for T267194: CloudVPS: enable TLS in openstack API endpoints: T319313: Prepare to block and throttle Openstack APIs.
Tue, Oct 4, 3:36 PM · cloud-services-team (Kanban), Patch-For-Review, Cloud-VPS
taavi edited parent tasks for T319313: Prepare to block and throttle Openstack APIs, added: T319312: Open Openstack APIs to the public internet; removed: T267194: CloudVPS: enable TLS in openstack API endpoints.
Tue, Oct 4, 3:36 PM · Patch-For-Review, cloud-services-team (Kanban), Cloud-VPS
taavi added a subtask for T319312: Open Openstack APIs to the public internet: T319313: Prepare to block and throttle Openstack APIs.
Tue, Oct 4, 3:36 PM · Patch-For-Review, cloud-services-team (Kanban)

Mon, Oct 3

taavi closed T319233: Pageviews API: Problems accessing data from python (requests) as Resolved.
Mon, Oct 3, 6:37 PM · Data-Engineering, Pageviews-API
taavi reopened T319233: Pageviews API: Problems accessing data from python (requests) as "Open".

https://meta.wikimedia.org/wiki/User-Agent_policy

Mon, Oct 3, 6:37 PM · Data-Engineering, Pageviews-API
taavi committed rCTVA8da432b1484c: Remove labstore* dump volumes (authored by taavi).
Remove labstore* dump volumes
Mon, Oct 3, 2:33 PM
taavi committed rCTVAa9dc97f821e0: Specify NFS node label selector (authored by taavi).
Specify NFS node label selector
Mon, Oct 3, 1:11 PM

Thu, Sep 29

taavi added projects to T318918: Undeploy patch to use old PHP serialization in PHP 7.4: SRE, serviceops.
Thu, Sep 29, 10:56 AM · Continuous-Integration-Infrastructure, serviceops, SRE
taavi added a comment to T261872: Drop PHP 7.2 & 7.3 support from MediaWiki master branch, once Wikimedia production is on 7.4.

Policy-required wikitech-l message: https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/thread/VMJXVVAQR6S74PR4M5LMW3EAW7VNM7D6/

Thu, Sep 29, 10:26 AM · MW-1.40-notes (1.40.0-wmf.4; 2022-10-03), MediaWiki-Releasing, PHP 7.3 support, serviceops, PHP 7.2 support

Wed, Sep 28

taavi added a comment to T318858: login.toolforge.org (tools-sgebastion-10) has broken emacs-gtk installed.

Both are only supposed to have -nox (the non-graphical one) installed instead of -gtk. The Puppet code pulls in the emacs meta package, and I suspect something changed between -10 and -11 were provisioned which changed which one apt will install by default. I'll send a patch to update that.

Wed, Sep 28, 8:03 PM · Toolforge
taavi added a comment to T317787: Pod Security Policies.

See also T279110: Replace PodSecurityPolicy in Toolforge Kubernetes, would be nice to keep Toolforge and PAWS on similar-ish technology stack.

Wed, Sep 28, 4:55 PM · PAWS
taavi added a comment to T318069: Migrate mwbot-rs tasks to Phabricator.

And tasks can either be in the top level "mwbot-rs" project, or in *one* of the subprojects (but not multiple), correct?

AIUI a task can be in the parent project, or *multiple* subprojects, but not both in the parent and a subproject.

Wed, Sep 28, 12:28 PM · mwbot-rs, Project-Admins
taavi added projects to T318345: Disable insecure rsa-ssh public key signature algorithm: cloud-services-team (Kanban), Security.

This will most likely cause some issues with at least some cloud vps/toolforge users so we probably want to announce this in advance on cloud-announce@.

Wed, Sep 28, 11:34 AM · Security, cloud-services-team (Kanban), Patch-For-Review, Infrastructure-Foundations

Tue, Sep 27

taavi added a comment to T301949: ToolsDB upgrade => Bullseye, MariaDB 10.4.

Anyhow, I suspect the bullseye replica would need to be re-cloned from the replica since it's been so long without being able to catch up on replication.

Tue, Sep 27, 7:13 PM · Data-Persistence (Consultation), Cloud-VPS (Debian Stretch Deprecation), cloud-services-team (Kanban), Toolforge, Data-Services
taavi added a comment to T301949: ToolsDB upgrade => Bullseye, MariaDB 10.4.

@taavi can you catch me up on the state of this task? Are still still mostly blocked by high ceph latency or did your comment at the end of June imply that you're past that?

Tue, Sep 27, 7:12 PM · Data-Persistence (Consultation), Cloud-VPS (Debian Stretch Deprecation), cloud-services-team (Kanban), Toolforge, Data-Services
taavi added a comment to T318687: Request creation of appservers VPS project.

Ok, sounds good. Mostly I'd been curious about the plan about other dependencies (databases, caches, ...) if it would've been the latter.

Tue, Sep 27, 1:59 PM · Cloud-VPS (Project-requests)
taavi added a comment to T318687: Request creation of appservers VPS project.

Hey. Is the plan to just test that the servers install all the packages properly or are you actually planning to test that reading/editing works properly?

Tue, Sep 27, 1:54 PM · Cloud-VPS (Project-requests)
taavi created T318713: `scap backport` should gracefully handle a single error when polling for patch merge.
Tue, Sep 27, 1:39 PM · Release-Engineering-Team, Scap
taavi created T318711: `scap backport` should display branch in the patch table.
Tue, Sep 27, 1:38 PM · Release-Engineering-Team, Scap
taavi awarded Blog Post: scap backport Makes Deployments Easy a Stroopwafel token.
Tue, Sep 27, 8:15 AM
taavi renamed T318639: PageTriage should mark articles that are very new from Incorporate NotSoFast user script to PageTriage should mark articles that are very new.
Tue, Sep 27, 8:03 AM · Patch-For-Review, PageTriage, Growth-Team
taavi edited projects for T318025: Unwikified thanks prompt, added: Thanks; removed WMF-General-or-Unknown.
Tue, Sep 27, 7:27 AM · Growth-Team, Thanks

Mon, Sep 26

taavi updated the task description for T271736: Migrate WMF production from PHP 7.2 to PHP 7.4.
Mon, Sep 26, 4:24 PM · Dumps-Generation, Patch-For-Review, Performance-Team (Radar), serviceops
taavi added a comment to T127367: Provide modern, non-NFS error log solution for Toolforge webservices and bots.

linking this here: https://wikitech.wikimedia.org/wiki/User:Majavah/Loki_notes

Mon, Sep 26, 11:53 AM · cloud-services-team (Kanban), Epic, Toolforge
taavi merged T318533: Offer S3-compatible storage into T276961: Support Openstack Swift APIs via the radosgw.
Mon, Sep 26, 11:47 AM · cloud-services-team (Kanban), Data-Services, Cloud-VPS, User-Marostegui
taavi added a subtask for T276961: Support Openstack Swift APIs via the radosgw: T289882: Q1:(Need By: TBD) rack/setup/install cloudswift100[12].
Mon, Sep 26, 11:47 AM · cloud-services-team (Kanban), Data-Services, Cloud-VPS, User-Marostegui
taavi added a parent task for T289882: Q1:(Need By: TBD) rack/setup/install cloudswift100[12]: T276961: Support Openstack Swift APIs via the radosgw.
Mon, Sep 26, 11:47 AM · SRE, Infrastructure-Foundations, ops-eqiad, netops, cloud-services-team (Hardware), DC-Ops
taavi merged task T318533: Offer S3-compatible storage into T276961: Support Openstack Swift APIs via the radosgw.
Mon, Sep 26, 11:46 AM
taavi added a comment to T318533: Offer S3-compatible storage.

Hi. I'll close this as a duplicate of T276961: Support Openstack Swift APIs via the radosgw, which tracks the overall project of providing some object storage service although the exact interfaces it'll be offering aren't yet known.

Mon, Sep 26, 11:46 AM
taavi closed T292465: Automate rebuild and rebuild toolsbeta-sgewebgrid-generic-0901 as Resolved.
Mon, Sep 26, 10:55 AM · cloud-services-team (Kanban), Patch-For-Review
taavi closed T292465: Automate rebuild and rebuild toolsbeta-sgewebgrid-generic-0901, a subtask of T290970: File System corruption on cloud-vps instances, as Resolved.
Mon, Sep 26, 10:55 AM · Cloud-Services-Worktype-Unplanned, Cloud-Services-Origin-Alert, User-dcaro, cloud-services-team (FY2021/2022-Q3), Cloud-VPS
taavi removed a project from T246563: 'webarchivebot' tool's custom webarchivebot-backend crashing because of code that expects /usr/sbin/sendmail: Toolforge.
Mon, Sep 26, 10:25 AM · Tools
taavi updated the task description for T301280: Move project-specific NFS mounts onto project-local NFS servers.
Mon, Sep 26, 7:00 AM · Patch-For-Review, Cloud-VPS, cloud-services-team (Kanban)
taavi closed T301295: Does the cloud-vps UTRS project need NFS?, a subtask of T301280: Move project-specific NFS mounts onto project-local NFS servers, as Resolved.
Mon, Sep 26, 6:59 AM · Patch-For-Review, Cloud-VPS, cloud-services-team (Kanban)
taavi closed T301295: Does the cloud-vps UTRS project need NFS? as Resolved.
Mon, Sep 26, 6:59 AM · Cloud-VPS, cloud-services-team (Kanban)

Sun, Sep 25

taavi created T318504: ENC API should update cloud/instance-puppet.git instead of requiring the caller to do so.
Sun, Sep 25, 6:18 PM · Cloud-VPS, cloud-services-team (Kanban)
taavi created T318503: ENC API should expose a JSON update interface.
Sun, Sep 25, 6:16 PM · Cloud-VPS, cloud-services-team (Kanban)
taavi added a project to T318479: Intermittent redis connection timeouts in Toolforge: cloud-services-team (Kanban).

Hmm. I've checked the Redis and Keepalived (what we use for providing HA for Redis) logs for the timeout on 24 September, nothing strange there. The Redis docs say that by default either the client nor the server have any idle timeout set, nor does our configuration set any limits.

Sun, Sep 25, 4:20 PM · cloud-services-team (Kanban), Toolforge

Sat, Sep 24

taavi closed T318481: MW 1.39.0-rc.0 tarball missing symfony/php73-polyfill as Resolved.
Sat, Sep 24, 6:09 PM · MW-1.39-notes, MediaWiki-Vendor, MW-1.39-release
taavi created T318481: MW 1.39.0-rc.0 tarball missing symfony/php73-polyfill.
Sat, Sep 24, 4:36 PM · MW-1.39-notes, MediaWiki-Vendor, MW-1.39-release
taavi closed T316975: cloud vps web proxy is down as Resolved.
Sat, Sep 24, 10:21 AM · Cloud-VPS, cloud-services-team (Kanban)

Fri, Sep 23

bd808 awarded T317438: Toolforge root for TheresNoTime a Meh! token.
Fri, Sep 23, 5:08 PM · cloud-services-team (Kanban), Toolforge
taavi closed T311911: Upgrade Toolforge Prometheus servers to Debian Bullseye or later, a subtask of T311897: Toolforge: migrate to Debian Bullseye or later, as Resolved.
Fri, Sep 23, 1:00 PM · Toolforge, cloud-services-team (Kanban), Epic
taavi closed T311911: Upgrade Toolforge Prometheus servers to Debian Bullseye or later, a subtask of T313030: Replace Toolschecker alerts with Prometheus based ones, as Resolved.
Fri, Sep 23, 1:00 PM · Toolforge, cloud-services-team (Kanban)
taavi closed T311911: Upgrade Toolforge Prometheus servers to Debian Bullseye or later as Resolved.
Fri, Sep 23, 1:00 PM · Toolforge, cloud-services-team (Kanban)
taavi closed T316860: Thanking in user rights log throws thanks-error-invalid-log-type as Resolved.
Fri, Sep 23, 12:59 PM · Growth-Team (Current Sprint), MW-1.40-notes (1.40.0-wmf.2; 2022-09-19), MW-1.39-release, Thanks

Thu, Sep 22

taavi triaged T318177: Global user IDs are not guaranteed to be stable as Lowest priority.

The "delete global account" feature was originally implemented for the purposes of the SUL migration and it hasn't really been used since. I don't think soft deletion is the way to go here.

Thu, Sep 22, 5:40 PM · MediaWiki-extensions-CentralAuth
taavi added a comment to T301295: Does the cloud-vps UTRS project need NFS?.

@Andrew looks like you removed the NFS server while they were still mounted on the existing instances? I fixed that by removing the NFS mounts and mounting the Cinder volume on utrs-production with a symlink on /home.

Thu, Sep 22, 5:14 PM · Cloud-VPS, cloud-services-team (Kanban)
taavi created T318360: Recommended solution for Terraform state backend.
Thu, Sep 22, 4:38 PM · Cloud-VPS, cloud-services-team (Kanban)
taavi closed T317438: Toolforge root for TheresNoTime as Resolved.

Ok, all done I think. Congrats!

Thu, Sep 22, 12:33 PM · cloud-services-team (Kanban), Toolforge

Wed, Sep 21

taavi claimed T317438: Toolforge root for TheresNoTime.

As far as I can tell this has been open for more than the minimum amount of time (1 week) with multiple supports and zero objections. I'll implement this in the next few days.

Wed, Sep 21, 7:44 PM · cloud-services-team (Kanban), Toolforge

Tue, Sep 20

taavi moved T318191: Evaluate opening the Wiki Replicas to the WAN (since we already have user authentication) from Inbox to Needs discussion on the cloud-services-team (Kanban) board.
Tue, Sep 20, 8:34 PM · cloud-services-team (Kanban), Data-Services
taavi added a project to T318191: Evaluate opening the Wiki Replicas to the WAN (since we already have user authentication): cloud-services-team (Kanban).
Tue, Sep 20, 8:34 PM · cloud-services-team (Kanban), Data-Services

Sun, Sep 18

taavi added a project to T318047: Collation problems on labsdbaccounts results in newly created Toolforge tools unable to connect to MariaDB databases: Data-Services.
Sun, Sep 18, 3:38 PM · User-dcaro, Data-Services, cloud-services-team (Kanban)
taavi added a comment to T318053: Renew toolserver.org Let’s Encrypt TLS certificate.

These are managed by acme-chief, and I see this in the logs:

Sep 18 15:33:00 tools-acme-chief-01 acme-chief-backend[4225]: Handling pushed CSR event for toolserver / rsa-2048
Sep 18 15:33:02 tools-acme-chief-01 acme-chief-backend[4225]: DNS server 208.80.154.135 (ACMEChallengeValidation.UNKNOWN) failed to validate challenge Challenge type: ACMEChallengeType.DNS01. _acme-challenge.toolserver.org TXT LPo11Ip3SBEw0nuVFz4oAsXKPo4mbfVrmFvCwwie1eA
Sep 18 15:33:02 tools-acme-chief-01 acme-chief-backend[4225]: Unable to validate challenge Challenge type: ACMEChallengeType.DNS01. _acme-challenge.toolserver.org TXT LPo11Ip3SBEw0nuVFz4oAsXKPo4mbfVrmFvCwwie1eA
Sun, Sep 18, 3:35 PM · cloud-services-team (Kanban), Toolforge
taavi edited projects for T318053: Renew toolserver.org Let’s Encrypt TLS certificate, added: cloud-services-team (Kanban); removed Traffic, Domains.
Sun, Sep 18, 3:34 PM · cloud-services-team (Kanban), Toolforge
taavi added a project to T316860: Thanking in user rights log throws thanks-error-invalid-log-type: MW-1.39-release.
Sun, Sep 18, 11:16 AM · Growth-Team (Current Sprint), MW-1.40-notes (1.40.0-wmf.2; 2022-09-19), MW-1.39-release, Thanks
taavi claimed T316860: Thanking in user rights log throws thanks-error-invalid-log-type.
Sun, Sep 18, 11:14 AM · Growth-Team (Current Sprint), MW-1.40-notes (1.40.0-wmf.2; 2022-09-19), MW-1.39-release, Thanks

Fri, Sep 16

taavi triaged T272436: wmr-bot home directory using high NFS storage as High priority.

Hello again. Your user account is again showing up in the list of largest disk space users with roughly 200G of data in its $HOME. What exactly is being stored here and how is it related to the Wikimedia movement?

Fri, Sep 16, 3:19 PM · Tools

Wed, Sep 14

taavi added a comment to T316975: cloud vps web proxy is down.
/var/log/nginx/error.log
2022/09/14 04:50:33 [alert] 556396#556396: worker process 1429067 exited on signal 6
2022/09/14 04:50:33 [alert] 556396#556396: shared memory zone "memstore" was locked by 1429067
ter process /usr/sbin/nginx -g daemon on; master_process on;: ./debian/modules/nchan/src/store/memory/memstore.c:720: nchan_store_init_worker: Assertion `procslot_found == 1' failed.
2022/09/14 04:50:33 [alert] 556396#556396: worker process 1429068 exited on signal 6
2022/09/14 04:50:33 [alert] 556396#556396: shared memory zone "memstore" was locked by 1429068
2022/09/14 04:50:33 [notice] 1429055#1429055: signal process started
ter process /usr/sbin/nginx -g daemon on; master_process on;: ./debian/modules/nchan/src/store/memory/memstore.c:720: nchan_store_init_worker: Assertion `procslot_found == 1' failed.
2022/09/14 05:20:21 [notice] 1430199#1430199: signal process started
Wed, Sep 14, 5:43 AM · Cloud-VPS, cloud-services-team (Kanban)

Tue, Sep 13

taavi added a comment to T317623: Request for changing LDAP (Wikitech) username.

@ayounsi @BCornwall Please don't if you don't fully understand the effects of an account rename on all the systems that use developer account/LDAP authentication.

Tue, Sep 13, 7:00 PM · SRE, LDAP-Access-Requests
taavi awarded T317438: Toolforge root for TheresNoTime a Love token.
Tue, Sep 13, 3:01 PM · cloud-services-team (Kanban), Toolforge
taavi updated subscribers of T317606: PHP Notice: Undefined index: asOfTime.

@Krinkle @Ladsgroup Hey. wmf.1 includes this change https://gerrit.wikimedia.org/r/c/mediawiki/core/+/829316 which looks related - could you have a look?

Tue, Sep 13, 7:11 AM · MW-1.40-notes (1.40.0-wmf.1; 2022-09-12), Patch-For-Review, Performance-Team, MediaWiki-libs-Rdbms
taavi triaged T317606: PHP Notice: Undefined index: asOfTime as Unbreak Now! priority.
Tue, Sep 13, 7:09 AM · MW-1.40-notes (1.40.0-wmf.1; 2022-09-12), Patch-For-Review, Performance-Team, MediaWiki-libs-Rdbms

Mon, Sep 12

taavi updated the task description for T309281: Remove prod-specific bits from cloud puppetmasters.
Mon, Sep 12, 10:14 AM · Infrastructure-Foundations, Puppet, Cloud-VPS, cloud-services-team (Kanban)

Sun, Sep 11

taavi updated the task description for T309281: Remove prod-specific bits from cloud puppetmasters.
Sun, Sep 11, 3:14 PM · Infrastructure-Foundations, Puppet, Cloud-VPS, cloud-services-team (Kanban)

Sat, Sep 10

taavi created T317480: Publish releases of the terraform provider.
Sat, Sep 10, 6:02 PM · Patch-For-Review, Cloud-VPS, cloud-services-team (Kanban)
taavi closed T253394: Horizon prefix puppet dashboard's "remove prefix" doesn't remove git change tracking files as Resolved.

I think I fixed this recently.

Sat, Sep 10, 4:51 PM · Horizon
taavi created T317478: Go library and terraform provider to work with the puppet enc API.
Sat, Sep 10, 4:50 PM · Cloud-VPS, cloud-services-team (Kanban)
taavi renamed T316463: Go library and terraform provider to work with the web proxy API from Go library to work with the web proxy API to Go library and terraform provider to work with the web proxy API.
Sat, Sep 10, 4:50 PM · Cloud-VPS, cloud-services-team (Kanban)
taavi created T317463: Rename references of labweb to cloudweb.
Sat, Sep 10, 8:36 AM · Patch-For-Review, cloud-services-team (Kanban)
taavi placed T206158: Investigate setting up HTTPS directly on beta appservers up for grabs.
Sat, Sep 10, 7:33 AM · User-Majavah, Beta-Cluster-Infrastructure
taavi closed T274666: Add keystone auth middleware to the puppet enc api, a subtask of T305453: Horizon should use a proxy to access cloud vps hosted apis, as Resolved.
Sat, Sep 10, 7:33 AM · Patch-For-Review, Horizon, cloud-services-team (Kanban)
taavi closed T274666: Add keystone auth middleware to the puppet enc api as Resolved.
Sat, Sep 10, 7:32 AM · Patch-For-Review, User-Majavah, cloud-services-team (Kanban), Cloud-VPS

Fri, Sep 9

taavi renamed T316982: High availability for the main cloud vps web proxy from Keepalived to provide high availability for the main cloud vps web proxy to High availability for the main cloud vps web proxy.
Fri, Sep 9, 9:15 AM · Patch-For-Review, Sustainability (Incident Followup), Cloud-VPS, cloud-services-team (Kanban)
taavi awarded T317393: Export and share alerts data a Love token.
Fri, Sep 9, 8:24 AM · Observability-Alerting, SRE
Legoktm awarded T317362: Move cloud-wide root keys to the main puppet repo a Like token.
Fri, Sep 9, 1:00 AM · Security, cloud-services-team (Kanban), Cloud-VPS

Thu, Sep 8

taavi merged T317363: Move cloud-wide root keys to the main puppet repo into T317362: Move cloud-wide root keys to the main puppet repo.
Thu, Sep 8, 7:56 PM · Security, cloud-services-team (Kanban), Cloud-VPS
taavi merged task T317363: Move cloud-wide root keys to the main puppet repo into T317362: Move cloud-wide root keys to the main puppet repo.
Thu, Sep 8, 7:56 PM · Security, cloud-services-team (Kanban), Cloud-VPS
taavi created T317363: Move cloud-wide root keys to the main puppet repo.
Thu, Sep 8, 7:55 PM · Security, cloud-services-team (Kanban), Cloud-VPS
taavi created T317362: Move cloud-wide root keys to the main puppet repo.
Thu, Sep 8, 7:55 PM · Security, cloud-services-team (Kanban), Cloud-VPS
taavi committed rLPRI277f7dcf3021: hieradata: add fake novaproxy passwords (authored by taavi).
hieradata: add fake novaproxy passwords
Thu, Sep 8, 7:44 PM

Wed, Sep 7

taavi added a comment to T317163: Scap backport should include the deployers username on the Gerrit approval comment.

Right now the +2 review message is "Approved via scap backport". How about something this instead?

Approved by dancy@deploy1002 using scap backport
Wed, Sep 7, 7:02 PM · Release-Engineering-Team (Bonus Level 🕹️), Scap
taavi added a comment to T317128: Failed to convert audio to MP3: /bin/bash: /usr/bin/lame: No such file or directory on Beta Cluster.

There's an instance called deployment-shellbox.deployment-prep.eqiad1.wikimedia.cloud - is that not used to run shellbox for beta?

Wed, Sep 7, 4:58 PM · Beta-Cluster-Infrastructure, Community-Tech, MediaWiki-extensions-Phonos
TheresNoTime awarded T317163: Scap backport should include the deployers username on the Gerrit approval comment a Like token.
Wed, Sep 7, 3:01 PM · Release-Engineering-Team (Bonus Level 🕹️), Scap
taavi updated the task description for T307357: Move cloud vps ns-recursor IPs to host/row-independent addressing.
Wed, Sep 7, 9:01 AM · SRE, netops, cloud-services-team (Kanban), Infrastructure-Foundations, Cloud-VPS

Tue, Sep 6

taavi created T317163: Scap backport should include the deployers username on the Gerrit approval comment.
Tue, Sep 6, 8:46 PM · Release-Engineering-Team (Bonus Level 🕹️), Scap

Sep 6 2022

taavi updated the task description for T317082: sre.hosts.downtime: add network devices support.
Sep 6 2022, 11:50 AM · Infrastructure-Foundations, SRE-tools
taavi updated the task description for T309281: Remove prod-specific bits from cloud puppetmasters.
Sep 6 2022, 10:13 AM · Infrastructure-Foundations, Puppet, Cloud-VPS, cloud-services-team (Kanban)

Sep 3 2022

taavi closed T316974: Toolforge membershhip not granted if duplicate requets declined as Invalid.

You need to log out and back in after the membership request was approved. This is a known bug in Striker (T144943) and is mentioned in the talk page message that we send out to all Toolforge users when their membership requests are approved.

Sep 3 2022, 3:27 PM · Toolforge
taavi claimed T316982: High availability for the main cloud vps web proxy.
Sep 3 2022, 3:18 PM · Patch-For-Review, Sustainability (Incident Followup), Cloud-VPS, cloud-services-team (Kanban)
taavi added a project to T316981: Monitoring for the main project-proxy instance going down: Sustainability (Incident Followup).
Sep 3 2022, 2:05 PM · Patch-For-Review, Sustainability (Incident Followup), Cloud-VPS, cloud-services-team (Kanban)
taavi added a project to T316982: High availability for the main cloud vps web proxy: Sustainability (Incident Followup).
Sep 3 2022, 2:05 PM · Patch-For-Review, Sustainability (Incident Followup), Cloud-VPS, cloud-services-team (Kanban)