As discovered in T237319, when responding with HTTP 304, the response of Special:EntityData contains body, which against HTTP specification.
nginx does seem to have problem handling this non-standard behaviour, which in turn bubbles up to issues at ATS level.
Acceptance criteria:
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T236988 ats-be on the text cluster is experiencing broken connections | |||
| Resolved | BUG REPORT | Ladsgroup | T237319 ATS serving 502 errors due to malformed responses from wikibase (HTTP 304s with message body content) | ||
| Resolved | Ladsgroup | T238901 Wikibase's Special:EntityData should not emit when responding with HTTP code 304 |
It's happening in EntityDataRequestHandler line 503. I don't know how to flush the response in mediawiki.
I had a dig around yesterday to figure out where something was being added to the body, but didn't see anything.
P9723 is simply the HTML of the pure "chrome" from the skin, without any content, right?
I would not be surprised is what Wikibase does is first generating HTML output of at least non-Wikibase parts of the page, and then figuring out to do with the request on the Wikibase end.
Change 552505 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Stop outputting anything in case of 304 responses in Special:EntityData
Change 552519 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.5] Stop outputting anything in case of 304 responses in Special:EntityData
Change 552505 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Stop outputting anything in case of 304 responses in Special:EntityData
Change 552519 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.5] Stop outputting anything in case of 304 responses in Special:EntityData
Mentioned in SAL (#wikimedia-operations) [2019-11-22T15:16:47Z] <ladsgroup@deploy1001> Synchronized php-1.35.0-wmf.5/extensions/Wikibase/repo/: Stop outputting anything in case of 304 responses in Special:EntityData (T238901) (duration: 00m 57s)
last BAD_INCOMING_RESPONSE error on cp1075 was a few seconds before deploying this:
20191122.15h16m44s CONNECT:[0] could not connect [BAD_INCOMING_RESPONSE] to 10.2.2.1 for 'https://appservers-rw.discovery.wmnet/wiki/Q76150152?action=constraintsrdf&nocache=1574435804338
it looks like it is solved, thanks!