Find out why recent 503 did not result in alerting for ORES

Closed, ResolvedPublic

Actions

See https://wikitech.wikimedia.org/wiki/Incident_documentation/20160610-ORES

I've confirmed that a request to "/v2/scores/testwiki/reverted/13458/" will produce a 503 response when the ORES service is stuck as it was last week.

The response content looks like this:

{
  "error": {
    "code": "server overloaded",
    "message": "Cannot process your request because the server is overloaded.  Try again in a few minutes."
  }
}

Here's the config in puppet: https://github.com/wikimedia/operations-puppet/blob/production/modules/nagios_common/files/check_commands/check_ores_workers

Here's the docs for check_http: https://www.monitoring-plugins.org/doc/man/check_http.html

https://gerrit.wikimedia.org/r/#/c/294309/