Page MenuHomePhabricator

Investigate MT timeouts error on cxserver
Closed, ResolvedPublic

Description

Such timeouts are frequently (at least couple of times daily) seen on cxserver logstash errors.

  • User impact: unknown
  • Log:
{
  "headers": {
    "content-type": "application/problem+json"
  },
  "stack": "HTTPError: ETIMEDOUT\n    at request.then (/srv/service/node_modules/preq/index.js:246:19)\n    at tryCatcher (/srv/service/node_modules/bluebird/js/release/util.js:16:23)\n    at Promise._settlePromiseFromHandler (/srv/service/node_modules/bluebird/js/release/promise.js:547:31)\n    at Promise._settlePromise (/srv/service/node_modules/bluebird/js/release/promise.js:604:18)\n    at Promise._settlePromise0 (/srv/service/node_modules/bluebird/js/release/promise.js:649:10)\n    at Promise._settlePromises (/srv/service/node_modules/bluebird/js/release/promise.js:725:18)\n    at _drainQueueStep (/srv/service/node_modules/bluebird/js/release/async.js:93:12)\n    at _drainQueue (/srv/service/node_modules/bluebird/js/release/async.js:86:9)\n    at Async._drainQueues (/srv/service/node_modules/bluebird/js/release/async.js:102:5)\n    at Immediate.Async.drainQueues [as _onImmediate] (/srv/service/node_modules/bluebird/js/release/async.js:15:14)\n    at runCallback (timers.js:705:18)\n    at tryOnImmediate (timers.js:676:5)\n    at processImmediate (timers.js:658:5)",
  "name": "HTTPError",
  "message": "ETIMEDOUT",
  "body": {
    "internalErr": "ETIMEDOUT",
    "internalURI": "http://api-rw.discovery.wmnet/w/api.php",
    "internalQuery": "{}",
    "internalMethod": "post",
    "detail": "ETIMEDOUT",
    "type": "internal_http_error",
    "internalStack": "Error: ETIMEDOUT\n    at Timeout.setTimeout [as _onTimeout] (/srv/service/node_modules/preq/index.js:15:27)\n    at ontimeout (timers.js:436:11)\n    at tryOnTimeout (timers.js:300:5)\n    at listOnTimeout (timers.js:263:5)\n    at Timer.processTimers (timers.js:223:10)"
  },
  "status": 504,
  "levelPath": "fatal/service-runner/unhandled"
}

Event Timeline

Followup error seen after several ETIMEDOUT on logstash:

Error: Translation with Apertium fr>es failed: HTTPError: ETIMEDOUT
    at /srv/service/lib/mt/Apertium.js:37:11
    at tryCatcher (/srv/service/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/srv/service/node_modules/bluebird/js/release/promise.js:547:31)
    at Promise._settlePromise (/srv/service/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/srv/service/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/srv/service/node_modules/bluebird/js/release/promise.js:725:18)
    at _drainQueueStep (/srv/service/node_modules/bluebird/js/release/async.js:93:12)
    at _drainQueue (/srv/service/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/srv/service/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues [as _onImmediate] (/srv/service/node_modules/bluebird/js/release/async.js:15:14)
    at runCallback (timers.js:705:18)
    at tryOnImmediate (timers.js:676:5)
    at processImmediate (timers.js:658:5)

Most of timeouts are happening with Apertium only at moment.

There are few other (unrelated to MT) timeouts:

request.params.0	   	/v2/suggest/source/Orxan Əliyev (federasiya rəhbəri)/az
request.query.sourcelanguages	ru,en,it,egl,gag
request.url	   	        /v2/suggest/source/Orxan%20%C6%8Fliyev%20(federasiya%20r%C9%99hb%C9%99ri)/az?sourcelanguages=ru%2Cen%2Cit%2Cegl%2Cgag
request_id	   	        876532f8-3903-4b6b-a603-e91be97cfcc6

This need to recheck with latest Apertium service migration.

This seems no longer happening in Apertium logs. Moving to Done.