Page MenuHomePhabricator

wbsearchentities now returns an error with type=lexeme
Closed, ResolvedPublic

Description

I have used action=wbsearchentities with type=lexeme in the Ordia Toolforge tool. It has worked but now it no longer works.

Here is a short Python program that reproduces the problem.

import requests

HEADERS = {
    'User-Agent': 'Ordia',
}

params = {
    'action': 'wbsearchentities',
    'format': 'json',
    'search': "ankomst",
    'language': "da",
    'type': 'lexeme',
}

response = requests.get(
'https://www.wikidata.org/w/api.php',
    headers=HEADERS, params=params)

print(response.ok)

Result: Prints False. And a message with "Our servers are currently under maintenance or experiencing a technical problem." and status code 500.

Expect: Prints True

Disabling "'type': 'lexeme'," yields a non-error result, - but does not search lexemes

I do not recall hearing of any changes in the API, and the documentation at https://www.wikidata.org/w/api.php?action=help&modules=wbsearchentities is still indicating type=lexeme should work.

The below code does not return an error, but does not return a result (it should return L45438).

params = {
    'action': 'wbsearchentities',
    'format': 'json',
    'search': "L:ankomst",
    'language': "da",
    #'type': 'lexeme',
}

Details

Related Gerrit Patches:
mediawiki/extensions/WikibaseLexemeCirrusSearch : wmf/1.34.0-wmf.3Fix reference to classes that moved
mediawiki/extensions/WikibaseLexemeCirrusSearch : masterFix reference to classes that moved

Event Timeline

Fnielsen created this task.May 2 2019, 9:15 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 2 2019, 9:15 AM
Restricted Application added a project: Discovery-Search. · View Herald TranscriptMay 2 2019, 11:20 AM

Another way to generate status code 500 is by, e.g.,

Result: Status 500 Internal Server Error.

Expected: Status 200.

Lucas_Werkmeister_WMDE triaged this task as Unbreak Now! priority.May 2 2019, 3:11 PM
Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptMay 2 2019, 3:11 PM
Tarrow added a subscriber: Tarrow.EditedMay 2 2019, 3:18 PM

/w/api.php?action=wbsearchentities&format=json&search=L:ankomst&language=da&type=lexeme PHP Fatal Error from line 205 of /srv/mediawiki/php-1.34.0-wmf.3/extensions/WikibaseLexemeCirrusSearch/src/LexemeSearchEntity.php: Class undefined: Wikibase\Repo\Search\Elastic\WikibasePrefixSearcher

MediaWiki or an installed extension requires this class but it is not embedded directly in MediaWiki's git repository and must be installed separately by the end user.

Please see <a href="https://www.mediawiki.org/wiki/Download_from_Git#Fetch_external_libraries">mediawiki.org</a> for help on installing the required components

Tarrow added a comment.May 2 2019, 3:29 PM

I suspect this is related to the recent moving of Elasticsearch code of of wikibase. e.g. https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/505313

Looking at logstash, I'm seeing this happen on mwdebug, so maybe folks are already looking into it. Anyway, here's a stacktrace if it's helpful

message
PHP Fatal Error: Class undefined: Wikibase\Repo\Search\Elastic\WikibasePrefixSearcher
trace
#0 /srv/mediawiki/php-1.34.0-wmf.3/extensions/WikibaseLexemeCirrusSearch/src/LexemeSearchEntity.php(205): NO_FUNCTION_GIVEN()
#1 /srv/mediawiki/php-1.34.0-wmf.3/extensions/Wikibase/repo/includes/Api/TypeDispatchingEntitySearchHelper.php(54): Wikibase\Lexeme\Search\Elastic\LexemeSearchEntity->getRankedSearchResults(string, string, string, integer, boolean)
#2 /srv/mediawiki/php-1.34.0-wmf.3/extensions/Wikibase/repo/includes/Api/SearchEntities.php(121): Wikibase\Repo\Api\TypeDispatchingEntitySearchHelper->getRankedSearchResults(string, string, string, integer, boolean)
#3 /srv/mediawiki/php-1.34.0-wmf.3/extensions/Wikibase/repo/includes/Api/SearchEntities.php(254): Wikibase\Repo\Api\SearchEntities->getSearchEntries(array)
#4 /srv/mediawiki/php-1.34.0-wmf.3/includes/api/ApiMain.php(1593): Wikibase\Repo\Api\SearchEntities->execute()
#5 /srv/mediawiki/php-1.34.0-wmf.3/includes/api/ApiMain.php(531): ApiMain->executeAction()
#6 /srv/mediawiki/php-1.34.0-wmf.3/includes/api/ApiMain.php(502): ApiMain->executeActionWithErrorHandling()
#7 /srv/mediawiki/php-1.34.0-wmf.3/api.php(87): ApiMain->execute()
#8 /srv/mediawiki/w/api.php(3): include(string)
#9 {main}

Change 507822 had a related patch set uploaded (by Tarrow; owner: Tarrow):
[mediawiki/extensions/WikibaseLexemeCirrusSearch@master] Fix reference to classes that moved

https://gerrit.wikimedia.org/r/507822

Change 507847 had a related patch set uploaded (by Tarrow; owner: DCausse):
[mediawiki/extensions/WikibaseLexemeCirrusSearch@wmf/1.34.0-wmf.3] Fix reference to classes that moved

https://gerrit.wikimedia.org/r/507847

Change 507822 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexemeCirrusSearch@master] Fix reference to classes that moved

https://gerrit.wikimedia.org/r/507822

Cherry-pick is failing with an error from the mediainfo tests (but they didn't fail on master); I feel nervous about over-riding the test result and pushing this into production. Can I get a sanity check?

The patch looks sane to me, and I think broken lexeme search is important enough that I’d be okay with overriding the gate-and-submit failure and force-merging the change.

alaa_wmde added a subscriber: alaa_wmde.EditedMay 3 2019, 9:20 AM

The failing browser test on media info extension seems to fail only on php 7.0 image.. we do not have that in production as far as we can tell. If that's the case, the risk of patching this in production seem quite low compared to cost of delaying this bugfix.

Let's:

  • patch in production
  • right afterwards, manually test that MediaInfo feature (edit captions) isn't broken in production (that's the failing browser test on deployment branch now)
  • meanwhile investigate MediaInfo browser failure (one possible thing here is that MediaInfo has some commit in their master that might also need to be cherry-picked to make this failure disappear on deployment branch)
  • right afterwards, manually test that MediaInfo feature (edit captions) isn't broken in production (that's the failing browser test on deployment branch now)

better yet, let’s test that during deployment, on the debug server (before syncing the patch to all other servers)

alaa_wmde added a comment.EditedMay 3 2019, 9:33 AM

right that's a lot better 👍

Change 507847 merged by Lucas Werkmeister (WMDE):
[mediawiki/extensions/WikibaseLexemeCirrusSearch@wmf/1.34.0-wmf.3] Fix reference to classes that moved

https://gerrit.wikimedia.org/r/507847

Mentioned in SAL (#wikimedia-operations) [2019-05-03T10:02:20Z] <lucaswerkmeister-wmde@deploy1001> Synchronized php-1.34.0-wmf.3/extensions/WikibaseLexemeCirrusSearch/: [[gerrit:507847|Fix reference to classes that moved (T222347)]] (duration: 00m 55s)

Lucas_Werkmeister_WMDE lowered the priority of this task from Unbreak Now! to High.May 3 2019, 10:03 AM
Lucas_Werkmeister_WMDE moved this task from incoming to in progress on the Wikidata board.

Deployed. I uploaded a new version of this file and added a caption to it on mwdebug1002 to confirm that this wasn’t broken. I guess gate-and-submit on the branch might just be broken :/

Adding to campsite board for verification, but I think we can lower the priority already.

Seems to be working again now. Thanks for all who helped!

Smalyshev lowered the priority of this task from High to Medium.May 3 2019, 7:17 PM
Jdforrester-WMF closed this task as Resolved.May 6 2019, 4:28 PM

Yes, it has been working fine now.