Page MenuHomePhabricator

Higher failed fetches error rate on some caching servers
Closed, ResolvedPublic

Description

Some frontend caching servers appear to have a way higher failed fetches error rate on upload than others

Event Timeline

jijiki created this task.Sep 18 2019, 11:01 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 18 2019, 11:01 AM
Vgutierrez moved this task from Triage to Caching on the Traffic board.Sep 18 2019, 11:04 AM

Change 537625 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] ATS: Ensure that the origin timeout is also applied to parent servers

https://gerrit.wikimedia.org/r/537625

Change 537625 merged by Vgutierrez:
[operations/puppet@production] ATS: Ensure that the origin timeout is also applied to parent servers

https://gerrit.wikimedia.org/r/537625

Change 537630 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] ATS: Avoid Proxy-Connection from spreading to varnish-fe and ats-be

https://gerrit.wikimedia.org/r/537630

Change 537630 merged by Vgutierrez:
[operations/puppet@production] ATS: Avoid Proxy-Connection from spreading to varnish-fe and ats-be

https://gerrit.wikimedia.org/r/537630

Mentioned in SAL (#wikimedia-operations) [2019-09-18T12:18:34Z] <vgutierrez> restarting ats-tls to avoid spreading Proxy-Connection header - T233205

Vgutierrez closed this task as Resolved.Sep 18 2019, 12:26 PM
Vgutierrez claimed this task.
Vgutierrez triaged this task as Normal priority.

Solved by preventing Proxy-Connection from spreading across varnish-fe and ats-be, thanks for reporting the issue @jijiki