Page MenuHomePhabricator

Filter out unsupported languages for cxserver
Open, MediumPublic

Description

cxserver should filter out unsupported language codes like qqq.

Error observed in cxserver logs are like:

MT processing error for: en > qqq. Error: Translation with Google failed for en > qqq. Error: 400 : Invalid Value

Event Timeline

This seems to be coming from the suggestions API where we pass the ULS most recently used languages.

Full error stack example:

MT processing error for: en > qqq. Error: Translation with Google failed for en > qqq. Error: 400 : Invalid Value
    at preq.post.then.catch (/srv/service/lib/mt/Google.js:79:10)
    at tryCatcher (/srv/service/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/srv/service/node_modules/bluebird/js/release/promise.js:547:31)
    at Promise._settlePromise (/srv/service/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/srv/service/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/srv/service/node_modules/bluebird/js/release/promise.js:725:18)
    at _drainQueueStep (/srv/service/node_modules/bluebird/js/release/async.js:93:12)
    at _drainQueue (/srv/service/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/srv/service/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues [as _onImmediate] (/srv/service/node_modules/bluebird/js/release/async.js:15:14)
    at runCallback (timers.js:705:18)
    at tryOnImmediate (timers.js:676:5)
    at processImmediate (timers.js:658:5)

Change 577050 had a related patch set uploaded (by Santhosh; owner: Santhosh):
[mediawiki/services/cxserver@master] SourceSuggester API: Filter out unsupported languages

https://gerrit.wikimedia.org/r/577050

Change 577050 merged by jenkins-bot:
[mediawiki/services/cxserver@master] SourceSuggester API: Filter out unsupported languages

https://gerrit.wikimedia.org/r/577050

To verify: Check from logstash, after deployment, that this error is gone.

Mentioned in SAL (#wikimedia-operations) [2020-03-12T06:14:02Z] <kart_> Updated cxserver to 2020-03-12-041806-production and added sectionmapping db config (T246316, T243430, T202276)

Looking at the 7 and 30 day graphs, I don't even see obvious reduction in the events. Given the stack traces it's impossible to say if there are other sources as well. However, I can confirm that neither
https://cxserver.wikimedia.org/v2?doc#!/Suggestions/get_v2_suggest_sections_title_from_to nor https://cxserver.wikimedia.org/v2?doc#!/Suggestions/get_v2_suggest_source_title_to give visible errors anymore.

@santhosh should this go to in progress again?

@santhosh should this go to in progress again?

yes