Requests to the citoid are repeatedly timing out with the error 504 gateway timeout.
Requests to both prod and staging servers of both citoid and zotero services, are timing out
i.e. curl -d '9791029801297' -H 'Content-Type: text/plain' https://staging.svc.eqiad.wmnet:4969/search
for zotero
https://en.wikipedia.org/api/rest_v1/data/citation/mediawiki/10.1038%2Fs41586-021-03470-x is an example of a request that timesout
For some reason this is not apparent on grafana in the 500 signal: https://grafana.wikimedia.org/d/NJkCVermz/citoid?orgId=1&refresh=5m&from=1708998234259&to=1709041434259
however a 503 signal is showing up on cxserver: https://grafana.wikimedia.org/d/F7rttgqmz/cxserver?orgId=1&refresh=30s&from=now-12h&to=now&viewPanel=15
500s
eqiad temporarily has been depooled