Page MenuHomePhabricator
Create Task
Maniphest T246316

Filter out unsupported languages for cxserver
Open, MediumPublic
Actions

Description

cxserver should filter out unsupported language codes like qqq.

Error observed in cxserver logs are like:

MT processing error for: en > qqq. Error: Translation with Google failed for en > qqq. Error: 400 : Invalid Value

Details

ProjectBranchLines +/-Subject
mediawiki/services/cxservermaster+7 -1SourceSuggester API: Filter out unsupported languages
Customize query in gerrit

Related Objects

Event Timeline

KartikMistry created this task.Feb 27 2020, 7:51 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 27 2020, 7:51 AM
Nikerabbit added a comment.Feb 27 2020, 7:53 AM

This seems to be coming from the suggestions API where we pass the ULS most recently used languages.

KartikMistry added a comment.Feb 27 2020, 11:45 AM

Full error stack example:

MT processing error for: en > qqq. Error: Translation with Google failed for en > qqq. Error: 400 : Invalid Value
    at preq.post.then.catch (/srv/service/lib/mt/Google.js:79:10)
    at tryCatcher (/srv/service/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/srv/service/node_modules/bluebird/js/release/promise.js:547:31)
    at Promise._settlePromise (/srv/service/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/srv/service/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/srv/service/node_modules/bluebird/js/release/promise.js:725:18)
    at _drainQueueStep (/srv/service/node_modules/bluebird/js/release/async.js:93:12)
    at _drainQueue (/srv/service/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/srv/service/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues [as _onImmediate] (/srv/service/node_modules/bluebird/js/release/async.js:15:14)
    at runCallback (timers.js:705:18)
    at tryOnImmediate (timers.js:676:5)
    at processImmediate (timers.js:658:5)
santhosh claimed this task.Mar 5 2020, 6:36 AM
gerritbot added a comment.Mar 5 2020, 6:36 AM

Change 577050 had a related patch set uploaded (by Santhosh; owner: Santhosh):
[mediawiki/services/cxserver@master] SourceSuggester API: Filter out unsupported languages

https://gerrit.wikimedia.org/r/577050

gerritbot added a project: Patch-For-Review.Mar 5 2020, 6:36 AM
santhosh triaged this task as Medium priority.Mar 5 2020, 6:37 AM
santhosh added a project: Language-Team (Language-2020-January-March).
santhosh moved this task from Backlog to In Review on the Language-Team (Language-2020-January-March) board.
gerritbot added a comment.Mar 10 2020, 7:43 AM

Change 577050 merged by jenkins-bot:
[mediawiki/services/cxserver@master] SourceSuggester API: Filter out unsupported languages

https://gerrit.wikimedia.org/r/577050

Maintenance_bot removed a project: Patch-For-Review.Mar 10 2020, 8:10 AM
Nikerabbit moved this task from In Review to Needs QA on the Language-Team (Language-2020-January-March) board.Mar 10 2020, 9:01 AM

To verify: Check from logstash, after deployment, that this error is gone.

Stashbot added a comment.Mar 12 2020, 6:14 AM

Mentioned in SAL (#wikimedia-operations) [2020-03-12T06:14:02Z] <kart_> Updated cxserver to 2020-03-12-041806-production and added sectionmapping db config (T246316, T243430, T202276)

Stashbot mentioned this in T243430: Basic service for mapping sections.Mar 12 2020, 6:14 AM
Stashbot mentioned this in T202276: Package apertium-pol-szl (Polish-Silesian).
Jpita added a subscriber: Jpita.Mar 16 2020, 3:32 AM

I still see the error in production https://logstash.wikimedia.org/app/kibana#/doc/logstash-*/logstash-syslog-2020.03.15/syslog?id=AXDgR0s3h3Uj6x1zv7Ur&_g=h@44136fa

Jpita moved this task from Needs QA to Pending input for QA on the Language-Team (Language-2020-January-March) board.Mar 16 2020, 3:35 AM
Nikerabbit added a comment.Mar 16 2020, 7:54 AM

Looking at the 7 and 30 day graphs, I don't even see obvious reduction in the events. Given the stack traces it's impossible to say if there are other sources as well. However, I can confirm that neither
https://cxserver.wikimedia.org/v2?doc#!/Suggestions/get_v2_suggest_sections_title_from_to nor https://cxserver.wikimedia.org/v2?doc#!/Suggestions/get_v2_suggest_source_title_to give visible errors anymore.

Jpita added a comment.Mar 17 2020, 8:52 AM

@santhosh should this go to in progress again?

santhosh added a comment.Mar 17 2020, 8:54 AM
In T246316#5974852, @Jpita wrote:

@santhosh should this go to in progress again?

yes

Jpita moved this task from Pending input for QA to In Progress on the Language-Team (Language-2020-January-March) board.Mar 17 2020, 8:55 AM
• Rileych edited projects, added Language-Team (Language-2020-Focus-Sprint); removed Language-Team (Language-2020-January-March).Mar 18 2020, 2:01 PM
• Rileych moved this task from Priority Backlog to In Progress on the Language-Team (Language-2020-Focus-Sprint) board.
Pginer-WMF edited projects, added Language-Team (Language-2020-January-March); removed Language-Team (Language-2020-Focus-Sprint).Mar 19 2020, 1:31 PM
• Rileych moved this task from Backlog to Compatibility on the CX-cxserver board.Jul 20 2020, 2:36 PM
• Rileych moved this task from Compatibility to Machine Translation on the CX-cxserver board.
santhosh removed santhosh as the assignee of this task.Mar 16 2021, 9:05 AM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL