Page MenuHomePhabricator

Exclude /metrics from TLS enforcement
Closed, ResolvedPublicBUG REPORT

Description

@bd808

I think that you are right. Those numbers don't look ok. And I do notice the following

curl -v  http://10.64.64.13:8000/metrics
* Expire in 0 ms for 6 (transfer 0x55f5c7bab0f0)
*   Trying 10.64.64.13...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55f5c7bab0f0)
* Connected to 10.64.64.13 (10.64.64.13) port 8000 (#0)
> GET /metrics HTTP/1.1
> Host: 10.64.64.13:8000
> User-Agent: curl/7.64.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Date: Mon, 18 Apr 2022 14:21:59 GMT
< Server: WSGIServer/0.2 CPython/3.7.3
< Content-Type: text/html; charset=utf-8
< Location: https://toolhub.wikimedia.org/metrics
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< X-Request-ID: f94727d19759410f8aba8f003e518c90
< Connection: close

which just doesn't look right. I wonder what prometheus does in this case.

which just doesn't look right. I wonder what prometheus does in this case.

That 301 redirect is basically the same issue that we saw with /healthz checks in T294072. I'll put up a patch to make the same fix for /metrics.

Event Timeline

bd808 changed the task status from Open to In Progress.Apr 18 2022, 3:52 PM
bd808 claimed this task.
bd808 triaged this task as High priority.

Change 783858 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[wikimedia/toolhub@main] config: Do not forge TLS for metrics collection

https://gerrit.wikimedia.org/r/783858

Change 783858 merged by jenkins-bot:

[wikimedia/toolhub@main] config: Do not force TLS for metrics collection

https://gerrit.wikimedia.org/r/783858

bd808 moved this task from Groomed/Ready to Review on the Toolhub board.

Change 786342 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[operations/deployment-charts@master] toolhub: Bump container version to 2022-04-21-215651-production

https://gerrit.wikimedia.org/r/786342

Change 786342 merged by jenkins-bot:

[operations/deployment-charts@master] toolhub: Bump container version to 2022-04-21-215651-production

https://gerrit.wikimedia.org/r/786342