Noted by @Krinkle just now in #wikimedia-releng:
14:56 <Krinkle> brennen: how sursprising is it to see wmf.19 code running today? 14:57 <brennen> er... 14:57 <Krinkle> apparently 5% of today's traffic is runnig wmf.19 according to https://performance.wikimedia.org/arclamp/svgs/daily/2022-07-25.excimer-wall.index.svgz?x=10.0&y=1429 14:57 <Krinkle> (the rest wmf.21) 14:57 <brennen> pretty surprising: https://versions.toolforge.org/ 14:57 <Krinkle> SAL says we switched group2 on 21st 14:58 <brennen> indeed i see some errors for .19 14:58 <brennen> i wonder if... something was depooled and didn't get synced on thursday? 14:58 <Krinkle> ok, please file task and escalate as you see fit. I'm OOO. Yeah, maybe ask SRE as well. 14:58 <brennen> ack, will do
Again, this should not be the case, since 1.39.0-wmf.21 (T308074) was rolled to all wikis on 2022-07-21.
Just for good measure:
21:03:16 brennen@deploy1002 /srv/mediawiki-staging (master u=) $ grep -c '[.]19' ./wikiversions.json 0