Page MenuHomePhabricator

TLS config issue for nginx on Buster
Closed, ResolvedPublic0 Estimated Story Points

Description

When running nginx on Buster with the TLS terminator config I can see the following error:

Jul 23 07:43:34 analytics-tool1004 systemd[1]: Starting A high performance web server and a reverse proxy server...
Jul 23 07:43:34 analytics-tool1004 nginx[27274]: nginx: [emerg] unknown directive "ssl_dyn_rec_enable" in /etc/nginx/nginx.conf:53
Jul 23 07:43:34 analytics-tool1004 nginx[27274]: nginx: configuration file /etc/nginx/nginx.conf test failed
elukey@analytics-tool1004:~$ sudo apt-cache policy nginx-full
nginx-full:
  Installed: 1.14.2-2
  Candidate: 1.14.2-2
  Version table:
 *** 1.14.2-2 500
        500 http://mirrors.wikimedia.org/debian buster/main amd64 Packages
        100 /var/lib/dpkg/status

There is still no wmf version of the nginx-full package for buster-wikimedia. In this case, the bit needed should be https://github.com/wikimedia/operations-software-nginx/blob/master/debian/patches/0100-dynamic-tls-records.patch

Event Timeline

If we need this to work ASAP, probably the most-expedient thing to do would be to patch our puppetization to exclude the patched features from config on buster only, and use the vendor package. Traffic is in the process of moving away from nginx, hopefully by EOQ-ish, after which we won't need the problematic custom package, and the stock vendor package should work fine for other uses of the tlsproxy module (but we're not quite ready enough, yet, to mess with our current solution by removing the WMF package from stretch!).

@BBlack thanks for the info! Not in a real rush, I was working on https://phabricator.wikimedia.org/T227860 to add TLS capabilities to the Analytics UIs, the only delay will be for Traffic :)

ema triaged this task as Medium priority.Jul 23 2019, 1:08 PM

Change 525483 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::tlsproxy::instance: avoid dynamic tls records in Buster

https://gerrit.wikimedia.org/r/525483

Change 525490 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] tlsproxy: toggle dynamic ssl_buffer_size settings

https://gerrit.wikimedia.org/r/525490

Change 525483 abandoned by Elukey:
profile::tlsproxy::instance: avoid dynamic tls records in Buster

Reason:
Race condition with Ema's patch https://gerrit.wikimedia.org/r/#/c/operations/puppet/ /525490/

https://gerrit.wikimedia.org/r/525483

Change 525490 merged by Ema:
[operations/puppet@production] tlsproxy: toggle dynamic ssl_buffer_size settings

https://gerrit.wikimedia.org/r/525490

Change 526147 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] tlsproxy: conditionally add ssl_ecdhe_curve to XCP

https://gerrit.wikimedia.org/r/526147

Change 526147 merged by Ema:
[operations/puppet@production] tlsproxy: conditionally add ssl_ecdhe_curve to XCP

https://gerrit.wikimedia.org/r/526147

The swap of Traffic for Traffic-Icebox in this ticket's set of tags was based on a bulk action for all such tickets that haven't been updated in 6 months or more. This does not imply any human judgement about the validity or importance of the task, and is simply the first step in a larger task cleanup effort. Further manual triage and/or requests for updates will happen this month for all such tickets. For more detail, have a look at the extended explanation on the main page of Traffic-Icebox . Thank you!

https://gerrit.wikimedia.org/r/c/operations/puppet/+/525490/ was merged long ago when traffic used nginx. Now that we don't use it, is this relevant? i.e. are there other teams still using nginx and Buster that require the patch?

https://gerrit.wikimedia.org/r/c/operations/puppet/+/525490/ was merged long ago when traffic used nginx. Now that we don't use it, is this relevant? i.e. are there other teams still using nginx and Buster that require the patch?

ssl_dyn_rec can be removed, it's no longer used/needed.

Change 881717 had a related patch set uploaded (by BCornwall; author: BCornwall):

[operations/puppet@production] tlsproxy: Remove ssl_dyn_rec support

https://gerrit.wikimedia.org/r/881717

Change 881717 merged by BCornwall:

[operations/puppet@production] tlsproxy: Remove ssl_dyn_rec support

https://gerrit.wikimedia.org/r/881717

Change 881902 had a related patch set uploaded (by BCornwall; author: BCornwall):

[operations/puppet@production] tlsproxy: Remove nginx_tune_for_media

https://gerrit.wikimedia.org/r/881902

BCornwall claimed this task.

ssl_dyn_rec has been removed entirely. Thanks for reporting!

Change 881902 merged by BCornwall:

[operations/puppet@production] tlsproxy: Remove nginx_tune_for_media

https://gerrit.wikimedia.org/r/881902