Page MenuHomePhabricator

Disable SSL 3.0 on Wikimedia sites to mitigate POODLE attack (CVE-2014-3566)
Closed, ResolvedPublic

Description

This POODLE bites: exploiting the SSL 3.0 fallback: http://googleonlinesecurity.blogspot.co.uk/2014/10/this-poodle-bites-exploiting-ssl-30.html
https://www.openssl.org/~bodo/ssl-poodle.pdf

The only workaround now is to disable SSL 3.0, but this will make IE6 users unable to access over HTTPS. If supporting IE6 is needed, how about we disable it for now and re-enable SSL 3.0 after TLS_FALLBACK_SCSV is available?


Version: wmf-deployment
Severity: normal

Details

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 3:55 AM
bzimport added projects: HTTPS, acl*sre-team.
bzimport set Reference to bz72072.
bzimport added a subscriber: Unknown Object (MLST).

This topic is being discussed on the Operations mailing list now.

(In reply to chmarkine from comment #0)

re-enable SSL 3.0 after TLS_FALLBACK_SCSV is available?

Quoting: "Google's TLS_FALLBACK_SCSV needs all clients and servers patched."

fixed misc-web-lb by restarting nginx on cp1043 and cp1044. they already had the right config but lacked that.

this fixed all the services behind misc-web, including gdash and ishmael

it did NOT fix Tool Labs

gerritadmin wrote:

Change 169978 had a related patch set uploaded by Chmarkine:
lists - disable SSLv3

https://gerrit.wikimedia.org/r/169978

gerritadmin wrote:

Change 169978 abandoned by Chmarkine:
lists - disable SSLv3

Reason:
I don't know if it will cause any problem if "ssl.use-sslv3" is not recognized by the current version. Anyway, there is no harm to wait until the server is upgraded.

https://gerrit.wikimedia.org/r/169978

(In reply to Daniel Zahn from comment #4)

it did NOT fix Tool Labs

fixed toollabs by

https://gerrit.wikimedia.org/r/#/c/169949/2

and restarting nginx on the instance tools-webproxy

https://www.ssllabs.com/ssltest/analyze.html?d=tools.wmflabs.org

also: https://gerrit.wikimedia.org/r/#/c/170117/2

Chmarkine, you had that nice table, what does your data say now, anything left?

(In reply to Daniel Zahn from comment #8)

Chmarkine, you had that nice table, what does your data say now, anything
left?

Thanks for fixing them.

I think the only two left are lists and dumps, but they are using so outdated versions of lighttpd that disabling SSL 3 is not an option. Is there any plan to upgrade these two servers?

[1] https://www.ssllabs.com/ssltest/analyze.html?d=lists.wikimedia.org
[2] https://www.ssllabs.com/ssltest/analyze.html?d=dumps.wikimedia.org

ticket for upgrading sodium: RT #5420
ticket for deploying new dumps misc hosts RT #4570
ticket for enabling https on dumps: RT #7067

ticket for enabling https on dumps: RT #7067

which is also T60292 and declined.

I think the only two left are lists and dumps, but they are using so outdated versions of lighttpd that disabling SSL 3 is not an option. Is there any plan to upgrade these two servers?

The plan would be to replace lighttpd with nginx. (per Faidon who recently already replace it in one other case)

@Dzahn, anything left actionable on this issue? Would you be willing to take it?

see my comment above:

ticket for upgrading sodium: RT #5420
ticket for deploying new dumps misc hosts RT #4570
ticket for enabling https on dumps: RT #7067

details would/should be on those tickets.

and no, sorry, not taking them

chasemp lowered the priority of this task from High to Medium.Jan 8 2015, 5:55 PM

see my comment above:

ticket for upgrading sodium: RT #5420
ticket for deploying new dumps misc hosts RT #4570
ticket for enabling https on dumps: RT #7067

details would/should be on those tickets.

and no, sorry, not taking them

OK thanks man, without an assignee then I am down grading priority here.

It seems dumps is now using the default cipher suite of nginx and SSL 3.0 is still enabled.
https://www.ssllabs.com/ssltest/analyze.html?d=dumps.wikimedia.org

I think dumps is the last server with SSL 3.0 enabled.

Change 188015 had a related patch set uploaded (by Yuvipanda):
dumps: Strengthen ssl settings

https://gerrit.wikimedia.org/r/188015

Patch-For-Review

I scanned all the public IP addresses in dns/templates/wikimedia.org, and found four SSL 3.0 enabled servers:

dataset1001	208.80.154.11
ms1001		208.80.154.16
radium		208.80.154.39
wikitech-static	166.78.57.240

https://wikitech-static.wikimedia.org/ is also vulnerable to OpenSSL CCS vulnerability.
https://www.ssllabs.com/ssltest/analyze.html?d=wikitech-static.wikimedia.org

Change 188015 merged by Yuvipanda:
dumps: Strengthen ssl settings

https://gerrit.wikimedia.org/r/188015

dumps.wikimedia.org reports an A now. what exactly is ms1001.wikimedia.org anyway? its ssl certificate calls it dumps.wikimedia.org

what exactly is ms1001.wikimedia.org anyway?

ms deprecated - media storage [1]

ms1001 is a nfs server of dumps and other datasets (dataset::nfs)
ms1001 is a rsyncer to internal peers of dumps (dataset::rsync::peers)
ms1001 is a rsyncer to the public of dumps (dataset::rsync::public)
ms1001 is a dumps.wikimedia.org (dumps)

[1] https://wikitech.wikimedia.org/wiki/Infrastructure_naming_conventions

Are radium and wikitech-static still in use? Is it needed to disable SSL3 on these two servers?

wikitech-static is a backup of wikitech in case the cluster breaks and wikitech becomes inaccessible, IIRC. Don't know about radium.

wikitech-static is still in use, but totally separate from puppet, running on a rackspace instance. radium is the tor-relay.

i fixed the SSL settings on wikitech-static. SSLv3 is disabled and i used the same cipher settings as on regular wikitech. feel free to re-scan now

So is radium needed to fix as well?

faidon subscribed.

radium is the Tor relay, not an HTTP server. It was due for a tor version upgrade anyway though, so I did this today.