I guess I was confused because the "Re-install" section was inside the "Quickstart" instructions. Are those instructions not supposed to be part of the Quickstart?

This was on a first run-through, or a re-install? (Mainly asking because the "run maintenance/update.php" bit is in the reinstall section, which maybe we should elaborate on.)

Still blocked as of this writing, not likely to move forward during this US workday. I'll see where things are at at 06:00 local when the cat wakes me up for breakfast.

May be related to T247562.

Seeing quite a bit of this, or what I think is this, in logstash over the last several hours.

Well, this is the latest release of Ubuntu and not some older LTS version. Do we want to drop support of the latest stable release of ubuntu as it would be "too old?

Just observed a spike of 361 of these in 1.35.0-wmf.23.

This appears to have been mitigated by the above patch. Unclear whether this task should remain open, but removing as train blocker.

Currently under discussion in #wikimedia-operations, seems likely a fix is inbound.

Per conversation in RelEng and input here, this doesn't seem severe enough to hold up the train (particularly over the weekend).

Given pre-existing nature of this breakage and per conversation in RelEng, we'll go ahead and remove this as a train blocker. It'd still definitely be good to reduce the log noise if possible.

Setting as UBN as long as this is marked a train blocker. Sounds like it's not serious in terms of real impact but it would be nice to eliminate logspam.

I'm guessing this was probably transient and isn't in fact a blocker, but someone should probably check me on that.

Thanks - and yeah, let's backport.

cc: @Tgr, @Anomie per that earlier ticket.

@Krinkle do T247484 or T247466 seem serious enough to prompt a rollback at this stage, or just block at group1?

Had one of these crop up in 1.35.0-wmf.23:

Proceeding.

Of course I said "14:00 UTC services window" when I meant 14:00 local, because timezones are hard and I'm bad at them.

Capturing some #wikimedia-operations IRC context:

This is an ongoing process, but we've got an established pairing routine and are making headway.

Group0 is currently on wmf.23.

I believe in production environment we use a backported ffmpeg and libvpx, but using a Debian 10 Buster base image should resolve this as well I think.

Yep.

Sorry for the Phabricator spam just now; thanks very much for collecting feedback. :)

In rPHEX8ed37ed2305b8823b292645b4841e2a68ec3b79f#6637335, @mmodell wrote:

In rPHEX8ed37ed2305b8823b292645b4841e2a68ec3b79f#6637304, @brennen wrote:

Overall this _looks_ sensible. Haven't been able to test locally. One note inline.

It's intentionally malformed json - a sort of common (though rudimentary) security measure. The purpose is to prevent it from being valid javascript so it can't be included with a <script src=> tag.

Will do.

I think I like this, assuming there's nothing that needs to be dynamic. Keeps people from having to clutter their global environments. The CLI from T246111 could probably generate an appropriate .env cheaply-ish.