Page MenuHomePhabricator

beta: Get SSL certificates for *.{projects}.beta.wmflabs.org
Closed, ResolvedPublic

Description

We now have nginx SSL proxies in front of the beta caches (Bug T38648). We still have to fix the certificate (that is *.wmflabs.org for now).

We need certificates generated by 'Labs CA' for the entries listed in role::protoproxy::ssl::beta and some more. I guess the easiest would be to create *.beta.wmflabs.org cert that will also contains the following DNS entries:

*.wikimedia.beta.wmflabs.org
*.wikibooks.beta.wmflabs.org
*.wikinews.beta.wmflabs.org
*.wikipedia.beta.wmflabs.org
*.wikiquote.beta.wmflabs.org
*.wikisource.beta.wmflabs.org
*.wikiversity.beta.wmflabs.org
*.wikivoyage.beta.wmflabs.org
*.wiktionary.beta.wmflabs.org

And the mobile ones:

*.m.wikimedia.beta.wmflabs.org
*.m.wikibooks.beta.wmflabs.org
*.m.wikinews.beta.wmflabs.org
*.m.wikipedia.beta.wmflabs.org
*.m.wikiquote.beta.wmflabs.org
*.m.wikisource.beta.wmflabs.org
*.m.wikiversity.beta.wmflabs.org
*.m.wikivoyage.beta.wmflabs.org
*.m.wiktionary.beta.wmflabs.org

*.zero.wikipedia.beta.wmflabs.org

See Also:
T56065: Enable HTTPS on a Labs instance
T70387: Beta Cluster no longer listens for HTTPS
T71269: Give all members of the deployment-prep project sudo

Details

Reference
bz48501
Related Gerrit Patches:
operations/puppet : productionbeta: Use Let's Encrypt cert
operations/mediawiki-config : masterdeployment-prep: Swap config to HTTPS

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 11 2015, 10:06 AM
greg added a comment.EditedAug 11 2015, 3:17 PM

@ArielGlenn: there's an NDA'd task at T97593 which Brandon is driving.

one more reason we should have this is to avoid needing https://gerrit.wikimedia.org/r/237523 (T105794, T112195)

Dzahn added a subtask: Restricted Task.Sep 11 2015, 12:01 AM

Let's Encrypt provides free trusted(*) DV non-wildcard certs. We have 31 domains lists here. If you think it's plausible, we can obtain 31 certs (one for each domain) from Let's Encrypt at zero cost.

(*) They will have their CA certificate cross-signed by IdenTrust next month, so the certs they issued won't be trusted until then.

I think we'd also want upload.beta.wmflabs.org, maybe stream.wmflabs.org, all of the m./zero. variants? What about mx.beta.wmflabs.org?

Chmarkine, there's always StartCom/StartSSL which has free certs, and they're already trusted by default in all major browsers.

Chmarkine added a comment.EditedSep 24 2015, 2:39 PM

Chmarkine, there's always StartCom/StartSSL which has free certs, and they're already trusted by default in all major browsers.

Yes. But StartCom charges $24.9 for each certificate revocation.

https://www.startssl.com/?app=43

Let's Encrypt provides free trusted(*) DV non-wildcard certs. We have 31 domains lists here. If you think it's plausible, we can obtain 31 certs (one for each domain) from Let's Encrypt at zero cost.
(*) They will have their CA certificate cross-signed by IdenTrust next month, so the certs they issued won't be trusted until then.

Let's Encrypt has received the cross-signature, so its certificates are now trusted by all major browsers.

[1] https://letsencrypt.org/2015/10/19/lets-encrypt-is-trusted.html

Let's Encrypt is in Public Beta now. Everyone can get free certificates from them now.

[1] https://letsencrypt.org/2015/12/03/entering-public-beta.html
[2] User Guide for getting certs using Let's Encrypt client: https://letsencrypt.readthedocs.org/en/latest/

Yeah, beta.wmflabs.org was in the private beta. Don't know if it can actually work with our setup though.

Dzahn added a subscriber: Mschon.

Would it be an option to flatten our subdomains?
We'd only need beta.wmflabs.org and *.beta.wmflabs.org to be in the certificate
(at e.g. DigiCert, those wildcards are $1425 for 3 years includes root and *)
For example:

  • beta.wmflabs.org
  • bits.beta.wmflabs.org
  • wikimedia.beta.wmflabs.org
  • commons-wikimedia.beta.wmflabs.org
  • wikipedia.beta.wmflabs.org
  • en-wikipedia.beta.wmflabs.org
  • nl-wikibooks.beta.wmflabs.org
  • en-m-wikibooks.beta.wmflabs.org
  • en-m-wikinews.beta.wmflabs.org

I've asked the same a few times and I don't remember the response. Can someone repeat why we can't just flatten the beta hostnames and just procure a *.beta.wmflabs.org certificate?

Tgr added a subscriber: Tgr.Jan 23 2016, 1:54 AM
Tgr added a comment.Jan 23 2016, 7:03 PM

Can someone repeat why we can't just flatten the beta hostnames and just procure a *.beta.wmflabs.org certificate?

Presumably because that would make beta less similar to production and less useful for testing things where the domain structure is relevant (e.g. CentralAuth shared cookies).

Paladox updated the task description. (Show Details)Mar 7 2016, 4:53 PM
Paladox added a subscriber: Paladox.
Krinkle removed a subscriber: Krinkle.Mar 8 2016, 10:02 PM
Krenair added a comment.EditedMar 13 2016, 2:16 AM

I'm going around a few tasks on this subject trying to merge/close everything together (this one, T70387, T75919, T97593). I'm going to see if I can get Let's Encrypt working for this.

kaldari removed a subscriber: kaldari.Mar 14 2016, 6:18 PM
Krenair closed subtask Restricted Task as Resolved.Apr 15 2016, 4:44 PM
Krenair added a comment.EditedApr 15 2016, 4:48 PM
In T97593#2115226, @Krenair wrote:

I am also wondering what the best way is to put the Let's Encrypt client onto the beta servers. Could we install it on the deployment-cache-text04 and deployment-cache-upload04 instances and have them get their own certificates? They're all running jessie and there is a letsencrypt package available.

In T97593#2116454, @Krenair wrote:

I checked this in deployment-cache-upload04 today. It's from jessie-backports and depends on lots of things from there, so you end up with something like:
sudo apt-get install letsencrypt python-cryptography=1.1.1-1~bpo8+1 python-pyasn1=0.1.9-1~bpo8+1 python-openssl=0.15.1-2~bpo8+1
(I did not actually allow this to continue, would like someone else to review since I'm not used to messing with package versions like this)

In T97593#2210712, @Krenair wrote:

ops have started using it in production for (ubuntu|apt|mirrors).wikimedia.org and it might be puppetised soon. (see T132450)

In T97593#2115226, @Krenair wrote:

I am also wondering what the best way is to put the Let's Encrypt client onto the beta servers. Could we install it on the deployment-cache-text04 and deployment-cache-upload04 instances and have them get their own certificates? They're all running jessie and there is a letsencrypt package available.

Ftr, I'd love to see us move in the direction of the let's encrypt model, where it's a straightforward, automated process for servers to get a valid cert for their identity. Whether it's LE specifically, or one of the projects that makes an LE-like api that organizations can run to automatically get certificates from their preferred CA.

I think we're going to need more of these (mysql and kafka servers / clients in the near term), and I'd like to get to the point where we're dealing with short lived certs wherever possible.

Ftr, I'd love to see us move in the direction of the let's encrypt model, where it's a straightforward, automated process for servers to get a valid cert for their identity. Whether it's LE specifically, or one of the projects that makes an LE-like api that organizations can run to automatically get certificates from their preferred CA.
I think we're going to need more of these (mysql and kafka servers / clients in the near term), and I'd like to get to the point where we're dealing with short lived certs wherever possible.

That might be hard as we're not exposing those servers directly to the internet. I suppose we could use an internal CA and run our own Let's Encrypt-like setup, but this is all far outside the scope of this task :)

It may or may not be a strict blocker but we should probably wait for T132812: Sort out letsencrypt puppetization for simple public hosts

Dzahn added a comment.Jun 1 2016, 11:37 PM

Its puppetized now, should not be hard anymore. We already use it in prod.

Krenair claimed this task.EditedAug 2 2016, 3:04 AM

This is now working for meta.wikimedia.beta.wmflabs.org and deployment.wikimedia.beta.wmflabs.org (and their mobile variants) as well as upload.beta.wmflabs.org. I'll try to handle all other domains tomorrow.

Change 247587 had a related patch set uploaded (by Alex Monk):
beta: Use Let's Encrypt for upload, and new self-signed SSL certificate for most of text

https://gerrit.wikimedia.org/r/247587

Dzahn awarded a token.Aug 2 2016, 3:40 AM
Dzahn rescinded a token.
Dzahn awarded a token.

Change 302409 had a related patch set uploaded (by Alex Monk):
deployment-prep: Swap config to HTTPS

https://gerrit.wikimedia.org/r/302409

Change 302409 merged by jenkins-bot:
deployment-prep: Swap config to HTTPS

https://gerrit.wikimedia.org/r/302409

That is really neat @AlexMonk-WMF ! Is there anything left to do?

Convince someone with ops rights to merge the patch

So that is now pending review / merge of https://gerrit.wikimedia.org/r/247587 beta: Use Let's Encrypt cert which is already on beta cluster.

Change 247587 merged by BBlack:
beta: Use Let's Encrypt cert

https://gerrit.wikimedia.org/r/247587

hashar closed this task as Resolved.Aug 31 2016, 3:15 PM
hashar added a subscriber: BBlack.

Thanks to @Krenair (with reviews from @BBlack ) we got SSL on beta cluster. That is quite an achievement.

The last remaining patch has been merged in. I believe there is nothing left to do there.

Can this ticket be made public nowadays? There is now T182927 and it seemed like a duplicate.

Edit: ignore this, this comment was meant to be on T97593)