Seeing 3659 of these (or pretty similar) since 03:30 UTC:

I'm assuming related:

Several of what I believe is the same issue logged in the last hour:

Thanks all. I'll roll this forward shortly.

I am making the assumption at this point that this one is safe to remove from train blockers, given likely volume of errors / impending hack. Please correct if I'm mistaken.

It's a non-trivial amount of logspam - some ~1000 log messages a minute when deployed to Group1 - and will make diagnosing other issues correspondingly more difficult if rolled out.

A fuller message in case the ErrorException is relevant:

Changing priority to UBN!, as this is listed as a train blocker.

Thanks for investigating!

I think probably not, but I'm also wondering if @Krinkle's objectcache: Use variadic signature for makeKey() might be relevant.

Not rolling back Group0 for now.

Looking through changelog, Aaron seems to have commit messages with cache-related stuff. Adding for that reason.

Setting UBN! and adding as a train blocker. Not rolling back Group0 for now.

Moving to UBN!, as this is a train blocker.

Does this just need us to update Heterogenous deployment/Train deploys?

Yep - my last comment here turns out to be wrong and I think this will work, getting a version of https://gerrit.wikimedia.org/r/c/mediawiki/core/+/508392 into shape.

See also: T216049: add CI job into operations/deployments-charts repo that helm lint packages and perform the helm index after merge.

As written originally:

Since we now have dev-images and Dockerfile contents don't live in the local-charts README, this should no longer be a problem.

See also: T216049

518333 checks that the configuration contains a top-level pipelines. A next step is turning the stage steps into individual classes with run() and validate() methods.

Planning to pair on this with @thcipriani this coming Friday.

I'm biased in favor of some non-JS solution partly because I'm not personally great at JS, but mostly because it feels like there's a lot of installation overhead there for something that needs to run easily on a client machine as part of a bootstrapping process. That said, I concur that we should give due consideration to languages already very well understood in our developer community.

Per conversation elsewhere, will pair with @thcipriani on the puppet role on Friday.

Just so I'm clear, in this instance is it sufficient to change workspace to /mnt/home/jenkins-deploy in the Jenkins config? I see that path exists.

How to change/update configuration

Cool. I'll push a patch for that shortly.

Taking a step back, I now wonder if we even need to install docker on host machines by default? Assuming people don't have to jump through any hoops to build images locally, which ideally should be the case...

Yeah, this seems worth exploring.

From docs:

We'll iterate on this, but it's working, at least.