| fgiunchedi | |
| Mar 16 2017, 9:15 AM |
| F8400067: image.png | |
| Jun 7 2017, 4:35 PM |
Swift is available over HTTPS internally now, mediawiki's FileBackend should be configured to use https instead of http. This could be a simple configuration change, though it was mentioned at the active/active datacenter meeting that Discovery-ARCHIVED had to do some mw change when activating https for search? (cc @Gehel @dcausse)
Done
Needs to contact both source and destination over https
Ditto, switch swift communication to https
Done
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | LSobanski | T111653 Encrypt all the things | |||
| Resolved | aaron | T160616 Enable HTTPS for swift clients |
When activating HTTPS for mediawiki -> elasticsearch traffic, we had to enable HTTP connection pooling to mitigate the SSL handshake overhead. This was particularly important for cross DC traffic (increased latency), where the handshake overhead was on the same order of magnitude than the actual payload. This is particularly important for search where a lot of requests are answered in < 10ms.
Connection pooling is not available out of the box and only supported by HHVM.
The code needed to enable connection pooling is not available in a generic fashion in mediawiki core but here is what we use for Cirrus:
https://github.com/wikimedia/mediawiki-extensions-CirrusSearch/blob/master/includes/Elastica/PooledHttp.php
Note that currently connection pools are configured in puppet and cannot be changed on the fly, this is currently done in puppet in modules/mediawiki/manifests/hhvm.pp (see namedPools in this file)
So in short if this is a high volume service connection pooling might be required in situations where mediawiki and swift run in different datacenter.
Drawbacks:
(Adding @EBernhardson to this ticket as he is the author of all of this)
SwiftFileBackend will need to force an https URL when it gets the storage_url back in the JSON auth response.
Change 345430 had a related patch set uploaded (by Aaron Schulz):
[mediawiki/core@master] Handle proxy-based TLS when placed in front of Swift
Change 345430 merged by jenkins-bot:
[mediawiki/core@master] Handle proxy-based TLS when placed in front of Swift
Change 355174 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[operations/mediawiki-config@master] Switch Swift URLs to HTTPs
Change 355174 merged by jenkins-bot:
[operations/mediawiki-config@master] Switch Swift URLs to HTTPs
@aaron you mentioned in your weekly notes that this had a minor performance effect. Positive or negative? How much are we talking about?
Deploy was 00:18 May 26 UTC, and
Looks like SwiftRepl is the last element of this task, and part of https://www.mediawiki.org/wiki/Wikimedia_Technology/Goals/2017-18_Q1#Performance
@fgiunchedi , how difficult does look to add to the replication script? I know you said that script needs some rewritting.
@aaron yeah it will need some love, in the meantime I've patched in https support so swiftrepl will DTRT.
Looks like this completes the list