Page MenuHomePhabricator

mobileapps 501: unsupported_language logspam
Closed, ResolvedPublic

Description

This:

root@scb1001:~# tail -1 /srv/log/mobileapps/main.log
{"name":"mobileapps","hostname":"scb1001","pid":2053,"level":50,"message":"501: unsupported_language","stack":"HTTPError: 501: unsupported_language\n    at parse (/srv/deployment/mobileapps/deploy/src/lib/parseDefinition.js:198:15)\n    at Request.<anonymous> (/srv/deployment/mobileapps/deploy/src/lib/parsoid-access.js:166:20)\n    at Request.tryCatcher (/srv/deployment/mobileapps/deploy/node_modules/bluebird/js/main/util.js:24:31)\n    at Promise._settlePromiseFromHandler (/srv/deployment/mobileapps/deploy/node_modules/bluebird/js/main/promise.js:582:31)\n    at Promise._settlePromiseAt (/srv/deployment/mobileapps/deploy/node_modules/bluebird/js/main/promise.js:727:18)\n    at Promise._settlePromises (/srv/deployment/mobileapps/deploy/node_modules/bluebird/js/main/promise.js:845:14)\n    at Async._drainQueue (/srv/deployment/mobileapps/deploy/node_modules/bluebird/js/main/async.js:79:16)\n    at Async._drainQueues (/srv/deployment/mobileapps/deploy/node_modules/bluebird/js/main/async.js:89:10)\n    at Async.drainQueues (/srv/deployment/mobileapps/deploy/node_modules/bluebird/js/main/async.js:14:14)\n    at doNTCallback0 (node.js:419:9)\n    at process._tickCallback (node.js:348:13)","status":501,"type":"unsupported_language","detail":"The language you have requested is not yet supported.","levelPath":"error/501","request_id":"2c3fcb4e-be9e-11e5-bf64-261415c16aec","msg":"501: unsupported_language","time":"2016-01-19T11:17:06.557Z","v":0}

is being spamming the logs quite a bit:

root@scb1001:~# grep -c unsupported_language /srv/log/mobileapps/main.log
13002
root@scb1001:~# head -1 /srv/log/mobileapps/main.log | jq .time
"2016-01-19T06:25:23.369Z"
root@scb1001:~# date
Tue Jan 19 11:19:51 UTC 2016

I have no idea if it's a real user-facing error or not, but in any case it's quite distracting.

Also, an error code in the 5xx range is probably not the right one for such an issue — this sounds like something that could fit in the 4xx range, possibly even a 404.

Event Timeline

faidon raised the priority of this task from to High.
faidon updated the task description. (Show Details)
faidon subscribed.

The Mobile Content Service emits this error for unsupported wiktionaries, which is basically all wiktionaries besides the english one. The reason this is happening is because RESTBase sends it a request for each wiktionary page render, so most of the requests fail.

The 501 code seems appropriate in this case, since the plan is to expand the functionality to other wiktionaries as well in due time.

PR #476 restricts RESTBase to ask for re-renders only for the English variant at this time, until we set a timeline for rolling out the support for other wiktionaries as well.

mobrovac claimed this task.

The PR has been merged and deployed. Resolving.

Reopening as the spam continues. There seem to be a lot of requests to the Mobile Content Service for wiktionaries, but we have no idea where they are coming from. Investigating...

Nevermind, false alarm. The logs appearing now are actually genuine in that there are some en.wiktionary entries which contain definitions for different languages, so the service groks.

Well… it shouldn't grok then :) We shouldn't be really throwing (loggable) server errors for things that are normal in nature.

@faidon, the improvements to the service that would minimise such log entries (and eliminate false positives) is tracked in T124267: Improvements for the /page/definition end point.

I have a patch in progress now to address at least part of this -- will look at T124267 as well before submitting.

Change 265508 had a related patch set uploaded (by Mholloway):
Further refinements to defn endpoint language checking and error responses

https://gerrit.wikimedia.org/r/265508

Change 265508 merged by Mobrovac:
Further refinements to defn endpoint language checking and error responses

https://gerrit.wikimedia.org/r/265508

The patch has been deployed in production. When no definition is found, 404s are returned now. They still end up in the logs with a full stack trace. I'll work on correcting that in the service-template-node as this is a general problem of the services based on it.

Change 265655 had a related patch set uploaded (by Mholloway):
Return definitions for all languages on whitelisted domain

https://gerrit.wikimedia.org/r/265655

Change 265755 had a related patch set uploaded (by Mobrovac):
Update to service-template-node v0.2.3

https://gerrit.wikimedia.org/r/265755

Change 265755 merged by Mholloway:
Update to service-template-node v0.2.3

https://gerrit.wikimedia.org/r/265755

The stack trace is no longer emitted now for errors with status !== 500. Also, the 501s have been turned into 404s for the time being, the underlying reason for it is being tackled in T124321: Wiktionary endpoint: return definitions for all languages within a supported domain.

Change 265655 merged by Mobrovac:
Return definitions for all languages on whitelisted domain

https://gerrit.wikimedia.org/r/265655

Change 266430 had a related patch set uploaded (by Mholloway):
Expand Wiktionary definition service coverage to reduce 404s

https://gerrit.wikimedia.org/r/266430

Change 266430 merged by Mobrovac:
Expand Wiktionary definition service coverage to reduce 404s

https://gerrit.wikimedia.org/r/266430

501s are mostly gone, stack traces do not appear any more in the logs for errors with status code !== 500, and the number of 404s has been reduced. Resolving.