Re: 304-vs-200, I was able to get some 304s, but only when I dropped the INM and relied on IMS. It seems like the ETags might be inconsistent between serial requests to ores for the same resource? (the LM timestamps are too a bit).
Tue, Mar 21
This is it. We're currently still testing/deploying the kernel that allows it to be enabled. After that we can do some testing/evaluation on BBR itself and report here. Our current thinking is we expect we'll enable it as the default congestion control for our public edge nodes, but probably not elsewhere in our network (app/db/etc in core DCs), as it's unclear whether it might lose to cubic in some corner cases on fast/local networks.
Fri, Mar 17
Thu, Mar 16
This was resolved on the server side back in early Dec when the MW config patch finally landed. There might be trailing traffic to this ticket in the form of reports/fixups of long-broken things, but there's really nothing left to actually do to finish decom.
Mon, Mar 13
Presently, we don't have any sort of "live" feed on security incidents other than what you've already mentioned. For some classes of incident, such a thing would be a liability.
Yes, the anomalies you're describing here were the result of a DDoS attack against us, and our mitigations to reduce user impact. The incident doc is private...
Wed, Mar 8
Some further thoughts that haven't been captured here:
Tue, Mar 7
Mon, Mar 6
Probably this should be merged into T133548, unless it's altered to be about implementing some other solution outside of that scope.
Fri, Feb 24
I think the wikitech discussion seems like it was, in fact, a good discussion of the issue. So if your goal is discussion, I don't see the issue here. The metawiki page does contain a lot of information that comes from the Operations team.
I really don't mean to be overly facile here, but if you're interested in having a discussion, we can have that at any usual public discussion venue. The wikitech mailing list might be a good start. Our IRC channels (e.g. Freenode #wikimedia-operations) work as well for informal, async discussion. Setting up a BOF or other sort of session on the topic at Wikimania might make sense as well. It would be helpful to understand better what sort of discussion you'd like to have. Is this a policy discussion about how the organization or Operations (or Finance) specifically weights green issues in various decisions in general? Or do you want to bring some information to the table that we might not be aware of about the evolving metrics of vendor green-ness and how to evaluate them? or.. ?
I think you can discuss that anywhere you like (within reason!).
This probably isn't the ideal location, but I can speak to the issue here since it's obvious that some will come looking here for that answer. The TL;DR is that environmental considerations were not a major factor in this decision. The basics of this decision go something like this:
Wed, Feb 22
Singapore approved by Legal and selected, which is pretty much our ideal candidate on a range of issues.
Feb 13 2017
Feb 10 2017
Yeah ipvsadm says "memory allocation problem" if you give it any kind of not-useful arguments (like delete a non-existent service, etc)
Feb 6 2017
I know a chunk of our rcstream traffic that I've observed in the past came from Google. I'm not sure who/what in Google drives that...
Feb 3 2017
Recording this while I remember it:
Jan 30 2017
We should probably divorce the RO/RW distinction from the core design here. Not all services will have an RW/RO distinction (I would expect most not to), and those will be things we try to eliminate with better (active/active) design over time. if a specific services needs a split into "active/passive RW + active/active RO", we can solve that by calling it two separate services at this level: foo-rw and foo-ro, with different active/passive rules and distinct failover.
Jan 27 2017
Jan 26 2017
cache_misc for this are all implemented and live now. The config declaration is now:
stream.wikimedia.org is part of cache_misc now, so if we have an expiring certificate here, I don't think we need to replace it.
What about benefactorevents / eventdonations?
Any updates here? What we're asking for here is a modern HTTPS-only configuration. I'd think an e-commerce vendor would be all about that...
Confirmed correct current operation:
- All HTTP access seems to redirect to HTTPS
- All HTTPS requests send response header: strict-transport-security: max-age=31536000; includeSubDomains; preload
Jan 25 2017
Jan 23 2017
Jan 11 2017
Jan 5 2017
It's not really my feature, I just happened to write the very short config patch to turn it on, because nobody else had at the time. For the history on this, see also:
T149858 , espectially from T87276#2055761 onwards, where some issues with Safari's support of both spellings was raised. Also the original discussion in https://meta.wikimedia.org/wiki/Research_talk:Wikimedia_referrer_policy , and the original code change for it here: https://gerrit.wikimedia.org/r/#/c/186104/2 . I'm not opposed to any particular path, but I'd be careful that someone fully research all the implications of any change to the previous misspelled variant, and possibly look at using the header instead of the meta tag.
These are now deployed (digicert in esams, globalsign elsewhere). Pending closing this until we document switching off either of the certs...
Dec 20 2016
Yes, we can do Q3. The actual work on our end is fairly minimal, just need to pencil it in and remember to get it done!
Dec 19 2016
In case such an incident happens before the changes in January and I'm not around, the procedure to switch GlobalSign to Digicert globally would be:
Status update - Digicert unified certs (RSA+ECDSA) are now deployed and stapled alongside the GlobalSign ones on all cache terminators. They're not being used for user traffic, but they're ready as a warm standby in the case that we need to deal with another issue like the past GlobalSign OCSP/revocation incident.
@Ppena (or anyone) - who's responsible in the WMF for store.wikimedia.org? This is a pretty basic request and it's been outstanding for months. It's one of the few non-confirming exceptions to our HTTPS policy remaining!
@EdErhart-WMF - Any update on setting the appropriate Strict-Transport-Security header on this service?
We're going to leave this as-is and assume eventstream replacement (which will be HTTPS-only from the get-go) will handle this for us.