Page MenuHomePhabricator

Cannot read property 'length' of undefined for definition
Closed, ResolvedPublic

Description

An error in definitions endpoint:

http://mobileapps.discovery.wmnet:8888/en.wiktionary.org/v1/page/definition/%D8%B4%D8%AF%D8%AF

{"status":500,"type":"internal_error","title":"TypeError","detail":"Cannot read property 'length' of undefined","method":"GET","uri":"/en.wiktionary.org/v1/page/definition/%D8%B4%D8%AF%D8%AF"}

Log:

[2019-08-14T17:32:10.328Z] ERROR: mobileapps/167 on scb1002: 500: internal_error (message="500: internal_error", status=500, type=internal_error, detail="Cannot read property 'length' of undefined", request_id=71e45f50-beb9-11e9-b5cb-132eff0d5cd8, levelPath=error/500)
    stack: TypeError: Cannot read property 'length' of undefined
        at exampleElements.map.filter.e (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/src/lib/definitions/parseDefinitions.js:86:35)
        at Array.filter (native)
        at constructDefinition (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/src/lib/definitions/parseDefinitions.js:86:14)
        at iterateOverPartsOfSpeechSections (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/src/lib/definitions/parseDefinitions.js:118:52)
        at languageHeadings.forEach (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/src/lib/definitions/parseDefinitions.js:153:9)
        at NodeList.forEach (native)
        at iterateOverLanguageSections (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/src/lib/definitions/parseDefinitions.js:152:22)
        at mUtil.createDocument.then.then (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/src/lib/definition.js:37:26)
        at tryCatcher (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/node_modules/bluebird/js/release/util.js:16:23)
        at Promise._settlePromiseFromHandler (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/node_modules/bluebird/js/release/promise.js:517:31)
        at Promise._settlePromise (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/node_modules/bluebird/js/release/promise.js:574:18)
        at Promise._settlePromise0 (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/node_modules/bluebird/js/release/promise.js:619:10)
        at Promise._settlePromises (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/node_modules/bluebird/js/release/promise.js:699:18)
        at _drainQueueStep (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/node_modules/bluebird/js/release/async.js:138:12)
        at _drainQueue (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/node_modules/bluebird/js/release/async.js:131:9)
        at Async._drainQueues (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/node_modules/bluebird/js/release/async.js:147:5)
        at Immediate.Async.drainQueues (/srv/deployment/mobileapps/deploy-cache/revs/615004fb84e0f2507c714aaec7891c8f628cdc60/node_modules/bluebird/js/release/async.js:17:14)
        at runCallback (timers.js:672:20)
        at tryOnImmediate (timers.js:645:5)
        at processImmediate [as _immediateCallback] (timers.js:617:5)
    --
    request: {
      "url": "/en.wiktionary.org/v1/page/definition/%D8%B4%D8%AF%D8%AF",
      "headers": {
        "user-agent": "curl/7.38.0",
        "x-request-id": "71e45f50-beb9-11e9-b5cb-132eff0d5cd8"
      },
      "method": "GET",
      "params": {
        "0": "/en.wiktionary.org/v1/page/definition/شدد"
      },
      "query": {},
      "remoteAddress": "127.0.0.1",
      "remotePort": 42030
    }

Event Timeline

Pchelolo created this task.Aug 14 2019, 5:32 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 14 2019, 5:32 PM
Mholloway added a subscriber: Mholloway.EditedAug 15 2019, 1:47 PM

I've proposed updating the definition endpoint to use lexemes from Wikidata (https://gerrit.wikimedia.org/r/#/c/mediawiki/services/mobileapps/+/530259/), which would resolve this incidentally.

Jhernandez triaged this task as Low priority.Wed, Aug 21, 3:42 PM
Jhernandez added a subscriber: Jhernandez.

We should discuss changing the implementation to Wikidata lexemes and the implications in a new task, to figure out the coverage compared to wiktionary.

For now, this seems like it would be another smaller fix for this task, making it low. If the error is happening very frequently we should raise the priority.

cc/ @Pchelolo for the frequency of the errors

The reasons I think we ought to move on to using Lexemes ASAP is that the current, Wiktionary-based results are:

  1. Very poor. They are blobs of arbitrary HTML from which clients are required to strip markup. In some cases, they even include messages between editors in the page text itself.
  2. Very expensive to create.
  3. Overly broad in scope. A little-known fact about the Wiktionary project is that each Wiktionary aims to provide a definition for every term in every language in its target language. Consequently, the English Wiktionary contains definitions for many thousands of terms like https://en.wiktionary.org/w/index.php?curid=6100340 for which there's a vanishingly small likelihood of an English-language user of the Wikipedia app (the sole consumer of this endpoint, as far as we know) ever running a search.
  4. Only available for the English Wiktionary.

I'll create a fix for this narrow bug in the meantime, but IMO the sooner we kill legacy definition processing, the better.

Change 532720 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/services/mobileapps@master] Definitions: Fix TypeError: Cannot read property 'length' of undefined

https://gerrit.wikimedia.org/r/532720

Change 532720 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Definitions: Fix TypeError: Cannot read property 'length' of undefined

https://gerrit.wikimedia.org/r/532720

Mentioned in SAL (#wikimedia-operations) [2019-08-27T17:41:50Z] <mholloway-shell@deploy1001> Started deploy [mobileapps/deploy@1869f79]: Fix definition endpoint TypeError (T230503)

Mentioned in SAL (#wikimedia-operations) [2019-08-27T17:46:30Z] <mholloway-shell@deploy1001> Finished deploy [mobileapps/deploy@1869f79]: Fix definition endpoint TypeError (T230503) (duration: 04m 39s)

Created T231367: Update /page/definitions to use lexemes from Wikidata about switching to use lexemes from Wikidata.

Mholloway closed this task as Resolved.Tue, Aug 27, 5:51 PM