Fri, Feb 19
See T266717 for some related discussion.
Thu, Feb 11
Fri, Feb 5
This will stop alerting when T262211 is resolved and relforge1003,1004 are in service. The alias is just a pointer to the elasticsearch::relforge role, so nothing further needs to be done once the role is applied.
Jan 21 2021
Nice find! Thanks for tracking this down.
Jan 11 2021
Jan 8 2021
No, I haven't had a chance to look at k8splay at all yet.
Jan 5 2021
Dec 18 2020
Doh, thanks @AntiCompositeNumber. :)
So, since the root cause here is a known issue, I'm duping this ticket over to the SVG rendering issue, but thanks @RoySmith for filing -- the subtasks are still open and will hopefully yield distinct improvements here.
Thanks to @CDanis for digging into this with me. There are a couple of different things going on.
I can't repro the CORS issue exactly, but I am getting a 503 from Varnish for https://upload.wikimedia.org/wikipedia/commons/thumb/b/ba/Circuit_de_la_Sarthe_track_map.svg/2880px-Circuit_de_la_Sarthe_track_map.svg.png ... but only on the 2880px- URL -- the smaller ones work fine. I suspect that's why @Aklapper couldn't immediately repro: my browser didn't try to fetch the 2880px edition until I resized the window big enough.
Thanks @Varnent for offering to look at this, as our primary contact with VIP. It turns out two other VIP-hosted domains, techblog.wikimedia.org and wikimediaendowment.org, also don't set an HSTS header.
Emailed Comms about it, will route this appropriately when I hear back.
@MPhamWMF You're all set! After we discussed a bit more, consensus is that you don't need to sign L3. It may take up to 30 minutes for the change to roll out everywhere.
You're all set! Nikolay, you should have received an automated email with the admin password. Yekaterina won't have received that email automatically; please share the admin password with her privately, or coordinate between yourselves to reset it to something new.
Merged! It'll be rolled out everywhere in 30 minutes, feel free to reopen if you need anything else.
Dec 17 2020
Great! Pardon a brief delay in getting you set up with the additional access @Ottomata mentioned -- you're the guinea pig for a new arrangement, so I'd like to get some feedback from European-time-zone colleagues on how to implement it, and we should be able to get it committed tomorrow.
Why, I'd swear that wasn't there yesterday...
Hi @MPhamWMF, welcome to the Foundation! Your wikitech username was indeed the right one (thanks!) and delightfully I used it to pull up your shell uid "mttp", just to complete the set. ;)
This is done:
Great! Feel free to reopen, or file a new ticket under Wikimedia-Mailing-lists, if you need anything else.
Perfect! That's done, then -- both list owners should have just received an automated email containing the new admin password. Please confirm you can use it to log in at https://lists.wikimedia.org/mailman/admin/wikimedia-ma, and then I'll close this ticket -- or, let me know if you need anything else. :)
Hi @Anass_Sedrati, happy to work on this. :)
Looks that way! I'll take care of this today, thanks for flagging.
Dec 16 2020
@thcipriani Based on the context in T250241 I'm guessing this already has your blessing (creating a releasers-mwcli group containing Jeena and Brennen), but do you mind approving here for the record? Then I'll get it rolling and merge https://gerrit.wikimedia.org/r/649958 first thing in the morning.
@toan You should be all set! The access group changes might take up to 30 min to roll out everywhere. Let me know if you have any trouble.
Already in the nda group:
@KFrancis Thank you!
Oh wow, I filed this and went to bed, love to wake up and see it fully handled. :) Thanks all!
Dec 15 2020
Quick correction -- this is now live on all appservers, but the old URL is still cached by the traffic layer for 24 hours or so. We could purge the cache if we were in a great hurry, but all things being equal, safer to let it expire -- I'll check back tomorrow and make sure this updated as expected.
Deployed and tested:
(P.S. Today is technology's department-wide Fun Day, so further progress from Tyler and me might not come until tomorrow; sorry for the inconvenience. Give a yell if this is urgent, and we can get it taken care of.)
Happened to see this go by -- I've dropped a single comment on the review, requesting a test be updated to match, and then I'll merge ASAP. Apologies for letting it stall for so long!
Dec 14 2020
NDA discussion is happening over at T269777, then this will be ready to go.
@toan In addition to the agreement you signed with WMDE, you'll need to sign another with WMF -- @KFrancis will get you set up. Once that's taken care of, I can go ahead on both this task and T269678.
@MattCleinman Thanks for the update! I've added you to the wmf group.
@Tchanders From the "WIP" in the title, I'm guessing this isn't ready for SRE to work on yet, so I'm assigning it back to you. Feel free to pass it to me when it's ready for action. If you have any questions or you'd like some help getting it filled out, just say the word!
Yep, the alert has cleared. Thanks!
Dec 8 2020
Dec 3 2020
Oh, that's a good idea! We'd set maintenance_host to the empty string before the switchover, so that no new jobs would start anywhere, then set it to the new hostname afterward.
Dec 2 2020
Dec 1 2020
Nov 24 2020
This is finished on all wikis, a little bit slower than our original schedule but not stretching into the holiday weekend as I'd feared. :)
Nov 23 2020
Nov 19 2020
The good news: s3 is also finished, so only enwiki is left, taking the longest as expected.
Nov 18 2020
You're probably already thinking about this, but just to make sure it's said out loud: mwmaint1002 is still running updateCollation for the ICU upgrade, and will be chewing through enwiki for some days, so best to leave mwmaint1002 in place until that's finished. (T264991)
@DAlangi_WMF Drive-by comment: If you only need Logstash, you don't need shell access. (Getting shell access involves a lot more steps for you the requester, so if you can get along without it, that makes things quicker and easier for you.)
s2, s6, and s7 have also finished. The s3 worker has completed wikis up through ruwikibooks (in alphabetical order).