Page MenuHomePhabricator

Begin execution of non-forward-secret ciphers deprecation
Closed, ResolvedPublic

Description

After reaching the <0.1% threshold we are going to deprecate the last non-FS cipher from our ciphersuite: AES128-SHA. To be able to do that we need to:

  • Gain knowledge about the remaining users of AES128-SHA
  • Put in place a communication plan to reach those users and make the change as less disruptive as possible
  • Design the deprecation plan
  • Execute the deprecation plan (on going, we will disable AES128-SHA support in our TLS termination layer on August 1st)

Details

Related Gerrit Patches:
operations/puppet : productionvarnish: get rid of AES128-SHA redirection to /sec-warning
operations/puppet : productionssl_ciphersuite: forward-secret only
operations/puppet : productionvcl: Bump AES128-SHA redirection to 100%
operations/puppet : productionvcl: Bump AES128-SHA pageview replacement to 10%
operations/puppet : productionvcl: Bump AES128-SHA pageview replacement to 4%
operations/puppet : productionbrowsersec: Fix italian translation
operations/puppet : productionvcl: disable show_diff for browsersec.inc.vcl
operations/puppet : productionvcl: fix html layout on browsersec
operations/puppet : productionvcl: use synthetic warning for 1% of AES128-SHA pageviews

Event Timeline

Vgutierrez triaged this task as Medium priority.Apr 19 2018, 2:42 PM
Vgutierrez created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 19 2018, 2:42 PM
Vgutierrez moved this task from Triage to TLS on the Traffic board.Apr 19 2018, 3:13 PM

After running several small captures (10 minutes lapses over 2 days), we've got the following results:

  • 56% MiTM victims
  • 32% deprecated human-operated UAs
  • 7% deprecated well behaved bots
  • 4% bad players (MiTM proxies, fake UAs, misbehaving bots)

These results are going to be verified with a 24h traffic capture cluster wide (T193376)

More details available here: https://docs.google.com/spreadsheets/d/1YcFM3sxd-zXcy6Sut0Cu9yRbaJJcEr8KQUHcKhd0DRY/edit?usp=sharing

Vgutierrez added a comment.EditedMay 7 2018, 3:06 PM

After completing T193376 and analyzing the gathered data, we've got the following results for 24h of traffic data beginning at 2018-05-03 16:57:

  • 47% MiTM victims (A)
  • 34% deprecated human-operated UAs (B)
  • 8% deprecated well behaved bots (C)
  • 9% bad players (MiTM proxies, fake UAs, misbehaving bots) (D)

Take into account that AES128-SHA represents currently a 0.0889% of our traffic for the last 30 days

Top 20 legit UAs (B+C)
Mozilla/5.0 (PLAYSTATION 3 4.82) AppleWebKit/531.22.8 (KHTML, like Gecko)
Nokia6280/2.0 (03.60) Profile/MIDP-2.0 Configuration/CLDC-1.1
Perl MediaWiki::Bot/5.006002 (https://metacpan.org/MediaWiki::Bot; [[User:Malarz pl]])
https://he.wikipedia.org/wiki/%D7%9E%D7%A9%D7%AA%D7%9E%D7%A9:KotzBot/3.15 (Unix 3.13.0.139; Mono 3.2.8; .NET CLR 4.0.30319.17020)
Nokia7610/2.0 (5.0509.0) SymbianOS/7.0s Series60/2.1 Profile/MIDP-2.0 Configuration/CLDC-1.0
SharpMediaWiki/1.2
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705;)
ZTE-Z432/1.0.3 NetFront/4.2 QTV5.1 Profile/MIDP-2.1 Configuration/CLDC-1.1
WikiHistory (http://de.wikipedia.org/wiki/Benutzer:APPER/WikiHistory)
https://he.wikipedia.org/wiki/%D7%9E%D7%A9%D7%AA%D7%9E%D7%A9:KotzBot/3.15 (Unix 3.13.0.141; Mono 3.2.8; .NET CLR 4.0.30319.17020)
DotNetWikiBot/3.15 (Unix 4.4.0.119; Mono 4.2.1; .NET CLR 4.0.30319.17020)
DotNetWikiBot/3.15 (Unix 3.13.0.139; Mono 3.2.8; .NET CLR 2.0.50727.1433)
DoCoMo/2.0 N01G(c500;TB;W24H16)
DoCoMo/2.0 N01F(c500;TB;W24H16)
Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36
Mozilla/5.0 (PLAYSTATION 3 4.81) AppleWebKit/531.22.8 (KHTML, like Gecko)
SAMSUNG-GT-C3322i/1.0 NetFront/4.2 Profile/MIDP-2.0 Configuration/CLDC-1.1
PeriodiBOT (Http: //es.wikipedia.org/wiki/Usuario_discusión:MarioFinale)
DoCoMo/2.0 P01G(c500;TB;W24H16)
DoCoMo/2.0 F07F(c500;TB;W24H16)
Mozilla/5.0 (Symbian/3; Series60/5.3 NokiaN8-00/111.040.1511; Profile/MIDP-2.1 Configuration/CLDC-1.1 ) AppleWebKit/535.1 (KHTML, like Gecko) NokiaBrowser/8.3.1.4 Mobile Safari/535.1 3gpp-gba
Top 20 non legit UAs (A+D)
Mozilla/5.0 (Windows NT 6.5; rv:35.0) Gecko/20100101 Firefox/35.0
Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko
Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.119 Safari/537.36
ProxySG Appliance
Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36
Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko
Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0
Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393
Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:59.0) Gecko/20100101 Firefox/59.0
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36
Mozilla/5.0 (iPad; CPU OS 11_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.0 Mobile/15E148 Safari/604.1
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36 Edge/15.15063
Vgutierrez updated the task description. (Show Details)May 7 2018, 3:10 PM
ema added a subscriber: ema.May 8 2018, 10:15 AM

Change 440114 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] vcl: use synthetic warning for 1% of AES128-SHA pageviews

https://gerrit.wikimedia.org/r/440114

Nirmos added a subscriber: Nirmos.Jun 14 2018, 3:41 PM

Mentioned in SAL (#wikimedia-operations) [2018-06-14T16:41:14Z] <vgutierrez> disable puppet on cache nodes before merging gerrit/440114 - T192555

Change 440114 merged by Vgutierrez:
[operations/puppet@production] vcl: use synthetic warning for 1% of AES128-SHA pageviews

https://gerrit.wikimedia.org/r/440114

Change 440372 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] vcl: fix html layout on browsersec

https://gerrit.wikimedia.org/r/440372

Change 440372 merged by Vgutierrez:
[operations/puppet@production] vcl: fix html layout on browsersec

https://gerrit.wikimedia.org/r/440372

Change 440375 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] vcl: disable show_diff for browsersec.inc.vcl

https://gerrit.wikimedia.org/r/440375

Change 440375 merged by Vgutierrez:
[operations/puppet@production] vcl: disable show_diff for browsersec.inc.vcl

https://gerrit.wikimedia.org/r/440375

Change 440380 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] browsersec: Fix italian translation

https://gerrit.wikimedia.org/r/440380

Change 440380 merged by Vgutierrez:
[operations/puppet@production] browsersec: Fix italian translation

https://gerrit.wikimedia.org/r/440380

Mentioned in SAL (#wikimedia-operations) [2018-06-14T19:21:47Z] <vgutierrez> Reenable puppet in cache:misc nodes - T192555

Mentioned in SAL (#wikimedia-operations) [2018-06-14T20:11:46Z] <bblack> re-enable and run puppet on text@codfw - T192555

Mentioned in SAL (#wikimedia-operations) [2018-06-14T20:21:19Z] <bblack> re-enable and run puppet on text@ulsfo - T192555

Mentioned in SAL (#wikimedia-operations) [2018-06-14T20:45:14Z] <bblack> re-enable and run puppet on rest of cache_text (eqiad, eqsin, esams) - T192555

Mentioned in SAL (#wikimedia-operations) [2018-06-14T21:12:40Z] <bblack> re-enable and run puppet on cache_upload - T192555

Change 441347 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] vcl: Bump AES128-SHA pageview replacement to 4%

https://gerrit.wikimedia.org/r/441347

Change 441347 merged by Vgutierrez:
[operations/puppet@production] vcl: Bump AES128-SHA pageview replacement to 4%

https://gerrit.wikimedia.org/r/441347

Change 441804 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] vcl: Bump AES128-SHA pageview replacement to 10%

https://gerrit.wikimedia.org/r/441804

Change 441804 merged by Vgutierrez:
[operations/puppet@production] vcl: Bump AES128-SHA pageview replacement to 10%

https://gerrit.wikimedia.org/r/441804

Change 444005 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] vcl: Bump AES128-SHA redirection to 100%

https://gerrit.wikimedia.org/r/444005

Change 444005 merged by Vgutierrez:
[operations/puppet@production] vcl: Bump AES128-SHA redirection to 100%

https://gerrit.wikimedia.org/r/444005

Mentioned in SAL (#wikimedia-traffic) [2018-07-09T09:02:11Z] <vgutierrez> Bump AES128-SHA traffic redirection to 100% - T192555

Vgutierrez updated the task description. (Show Details)Jul 30 2018, 1:06 PM

Change 449747 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] ssl_ciphersuite: forward-secret only

https://gerrit.wikimedia.org/r/449747

Change 449747 merged by Vgutierrez:
[operations/puppet@production] ssl_ciphersuite: forward-secret only

https://gerrit.wikimedia.org/r/449747

Mentioned in SAL (#wikimedia-operations) [2018-08-01T16:41:51Z] <vgutierrez> Turn off AES128-SHA support - T192555 && T147202

Change 450020 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] varnish: get rid of AES128-SHA redirection to /sec-warning

https://gerrit.wikimedia.org/r/450020

Is this now Resolved?

Change 450020 merged by Vgutierrez:
[operations/puppet@production] varnish: get rid of AES128-SHA redirection to /sec-warning

https://gerrit.wikimedia.org/r/450020

Jdforrester-WMF closed this task as Resolved.Aug 16 2018, 9:11 PM
Jdforrester-WMF assigned this task to Vgutierrez.

Please re-open if I'm wrong.