Similar cropping up a few times in 1.35.0-wmf.4:

Noting here that this is still cropping up a fair amount as of 1.35.0-wmf.4:

My sense is that this may or may not block tomorrow's promotion to group2, but probably does not warrant rolling back group1 at present?

This remains pretty noisy in production:

Beautiful, thanks!

This class of error has led to a lot of noise while tailing logs during this week's train deploy:

Will do!

I published new docker images, updated job, and re-ran this:

It's hitting end-of-workday here and I'm a bit fried, will pick up WIP patch in the morning.

Unless I'm mistaken, this should just take adding a package in the list here:

During planning discussions, it came up that we'd like to have this session happen prior to T235372, on the theory that there's a natural progression from one to the other.

This train is no longer blocked, and can proceed according to schedule. Handing off to @LarsWirzenius.

Deployed.

Only groups 0-1 are affected right now, so enwiki is fine.

Seeing quite a bit of this in production logs at present:

I intended to deploy, but:

Deployed to group0, updated roadmap. Will reassign task to Lars at EOD.

I can co-lead here as well.

I'd say I usually get a week or so, but it feels like it's happened every other day or so lately. Latest release channel Firefox / Debian, mostly on a single machine.

@Quiddity thoughts here?

@greg @josephine_l @jeena: I think multiple sessions is probably a fine idea, and those seem like reasonable starting points for discussion.

@josephine_l I'm primarily attending as a note taker, and should probably focus the time I'm not doing that on local dev and CI sessions. At any rate, I suspect there are considerably better-informed folks than me to lead this one...

One suggestion for a focus: Maintenance & cleanup of existing docs. This has come to mind thinking about docs and new developer onboarding - there's a ton of material but it's not always clear how it relates to the current state of things or how to navigate it. Technical debt lives in the documentation as much as anywhere...

Jeena's comment above covers it (edit: "it" being recent RelEng work) pretty well. We have a local k8s proof-of-concept with Minikube that I think could be used for basic MediaWiki development right now, but the work has led to questions about complexity both for users of the development environment and for maintainers of the environment itself.

It's likely worth mentioning explicitly that we've landed on Argo, after building out and evaluating proofs-of-concept for our final three candidates (Argo, Zuul v3, GitLab CI).

Shall we declare this Resolved?