• Certificate management. How do we make sure both proxies have access to the certificates? Do we need to (maybe we can run letsencrypt on both hosts)?

According to my reading of letsencrypt/ACME design , it should be possible to only share the agent certificate, as 'confirm I am the owner of this domain' and 'please sign this certificate' are distinct steps.

The current letsencrypt code does not allow this, but the required changes seem to be minor. A simple hack, replacing

https://github.com/letsencrypt/letsencrypt/blob/master/letsencrypt/client.py#L224

by

authzr = [messages.AuthorizationResource(uri="", new_cert_uri="https://acme-staging.api.letsencrypt.org/acme/new-cert", body=None)]

works. I'm not entirely sure how to fit this in the letsencrypt plugin framework, though...

In general, I'd prefer if we discourage using such custom domains – IMHO they open a can of worms (trademarks, DNS management, certificate mangement, TOS, etc.) that is not worth … well, what do they bring exactly? If there is a use case for a separate domain, then it is probably also worth to buy a "proper" certificate and change it every year.

I haven't looked into Let's Encrypt, but AFAIUI it sets up a challenge-responser at the webserver at a (customizable?) URL. So the SSL connections could be terminated at the proxy and simple http requests passed on, like in the current setup for Tools. Only the proxy would need to create/sign/update certificates. There is no need to maintain the certificate at the "application" web server as well, nor would it work, as (any) proxy would break the encrypted end-to-end connection.

If the connection from the proxy to the application web server would need to be encrypted or the application web server would need to imagine that is serving https for redirects & Co. to work, we could use separate, self-signed certificates for those, i. e. Internet <- Let's Encrypt certificate -> proxy <- self-signed certificate -> application web server.

But, again :-), I'd prefer if we only use custom domains for cases where there is something to gain from them.

Yes, I agree. Nonetheless, the current situation is that we already have custom domains, but implemented in an ad-hoc manner. Having a central end-point with a puppetized configuration makes it much easier to keep track of these domains.

In general, the SSL certificates would live on the proxy, and the backend services wouldn't need to change their configuration as compared to the current proxy setup. Whether those certificates are letsencrypt certs or manually-handled certs doesn't matter too much; the virtualhosts/proxying configuration would be the same.

I also don't think we should permit use of custom domains.