Page MenuHomePhabricator
Create Task
Maniphest T365692

PHP Notice: Undefined index: lexeme_language / lexical_category
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error

In line 231:

labels.normalized_message
[{reqId}] {exception_url}   PHP Notice: Undefined index: lexeme_language

In line 232:

labels.normalized_message
[{reqId}] {exception_url}   PHP Notice: Undefined index: lexeme_category
error.stack_trace
from /srv/mediawiki/php-1.43.0-wmf.6/extensions/WikibaseLexemeCirrusSearch/src/LexemeFulltextResult.php(231)
#0 /srv/mediawiki/php-1.43.0-wmf.6/extensions/WikibaseLexemeCirrusSearch/src/LexemeFulltextResult.php(231): MWExceptionHandler::handleError(integer, string, string, integer, array)
#1 /srv/mediawiki/php-1.43.0-wmf.6/extensions/CirrusSearch/includes/Search/MSearchResponses.php(88): Wikibase\Lexeme\Search\Elastic\LexemeFulltextResult->transformElasticsearchResult(Elastica\ResultSet)
#2 /srv/mediawiki/php-1.43.0-wmf.6/extensions/CirrusSearch/includes/Searcher.php(397): CirrusSearch\Search\MSearchResponses->transformAsResultSet(Wikibase\Lexeme\Search\Elastic\LexemeFulltextResult, string)
#3 /srv/mediawiki/php-1.43.0-wmf.6/extensions/CirrusSearch/includes/Searcher.php(218): CirrusSearch\Searcher->searchTextInternal(string)
#4 /srv/mediawiki/php-1.43.0-wmf.6/extensions/CirrusSearch/includes/CirrusSearch.php(305): CirrusSearch\Searcher->search(CirrusSearch\Search\SearchQuery)
#5 /srv/mediawiki/php-1.43.0-wmf.6/extensions/CirrusSearch/includes/CirrusSearch.php(278): CirrusSearch\CirrusSearch->searchTextReal(CirrusSearch\Search\SearchQuery)
#6 /srv/mediawiki/php-1.43.0-wmf.6/includes/search/SearchEngine.php(100): CirrusSearch\CirrusSearch->doSearchText(string)
#7 /srv/mediawiki/php-1.43.0-wmf.6/includes/search/SearchEngine.php(192): SearchEngine->{closure}()
#8 /srv/mediawiki/php-1.43.0-wmf.6/includes/search/SearchEngine.php(101): SearchEngine->maybePaginate(Closure)
#9 /srv/mediawiki/php-1.43.0-wmf.6/includes/specials/SpecialSearch.php(470): SearchEngine->searchText(string)
#10 /srv/mediawiki/php-1.43.0-wmf.6/includes/specials/SpecialSearch.php(246): MediaWiki\Specials\SpecialSearch->showResults(string)
#11 /srv/mediawiki/php-1.43.0-wmf.6/includes/specialpage/SpecialPage.php(719): MediaWiki\Specials\SpecialSearch->execute(NULL)
#12 /srv/mediawiki/php-1.43.0-wmf.6/includes/specialpage/SpecialPageFactory.php(1680): MediaWiki\SpecialPage\SpecialPage->run(NULL)
#13 /srv/mediawiki/php-1.43.0-wmf.6/includes/actions/ActionEntryPoint.php(502): MediaWiki\SpecialPage\SpecialPageFactory->executePath(string, MediaWiki\Context\RequestContext)
#14 /srv/mediawiki/php-1.43.0-wmf.6/includes/actions/ActionEntryPoint.php(145): MediaWiki\Actions\ActionEntryPoint->performRequest()
#15 /srv/mediawiki/php-1.43.0-wmf.6/includes/MediaWikiEntryPoint.php(200): MediaWiki\Actions\ActionEntryPoint->execute()
#16 /srv/mediawiki/php-1.43.0-wmf.6/index.php(58): MediaWiki\MediaWikiEntryPoint->run()
#17 /srv/mediawiki/w/index.php(3): require(string)
#18 {main}
Impact
Notes

Seems to be a new issue in 1.43.0-wmf.6

AC:

  • The search indices should be updated with the appropriate fields related to lexemes
  • The lexemes should be re-indexed to ensure that the fields are properly indexed

Details

MediaWiki Version
1.43.0-wmf.6
Request URL
https://www.wikidata.org/w/index.php?go=*&ns0=*&search=*&title=*
SubjectRepoBranchLines +/-
cirrus-streaming-updater: use latest imageoperations/deployment-chartsmaster+1 -1
search: add missing lexeme fieldsschemas/event/primarymaster+99 -0
Workaround missing lemma fieldsmediawiki/extensions/WikibaseLexemeCirrusSearchmaster+3 -2
Customize query in gerrit

Related Objects

Event Timeline

Aklapper created this task.May 23 2024, 11:17 AM
Maintenance_bot added a project: Wikidata.May 23 2024, 11:29 AM
Restricted Application added a project: wmde-wikidata-tech. · View Herald TranscriptMay 23 2024, 11:29 AM
Lucas_Werkmeister_WMDE added a project: Discovery-Search.EditedMay 23 2024, 11:53 AM
Lucas_Werkmeister_WMDE subscribed.

I think this is actually the same issue as T365684: Particular lexeme (L1326823) not indexed so search with the Wikidata API returns nothing – the full request URL (from Logstash) shows that it’s a search for “markedsmodning”.

Lucas_Werkmeister_WMDE moved this task from Incoming to Other Projects on the wmde-wikidata-tech board.May 23 2024, 12:02 PM
thcipriani moved this task from Untriaged to May 2024 on the Wikimedia-production-error board.May 23 2024, 3:39 PM
Fnielsen subscribed.May 24 2024, 10:39 AM

I now see that this is not just for a particular lexeme but for newly created (Danish?) lexemes, e.g., "dyreaktivist" https://www.wikidata.org/w/index.php?go=G%C3%A5&search=L%3Adyreaktivist&search=L%3Adyreaktivist&title=Special%3ASearch&ns0=1 see "dyreaktivist (L1326971)"

Gehel triaged this task as High priority.May 27 2024, 12:41 PM
Gehel moved this task from needs triage to Current work on the Discovery-Search board.
Gehel edited projects, added Discovery-Search (Current work); removed Discovery-Search.
dcausse claimed this task.May 28 2024, 8:46 AM
dcausse moved this task from Incoming to In Progress on the Discovery-Search (Current work) board.
gerritbot added a comment.May 28 2024, 8:50 AM

Change #1036566 had a related patch set uploaded (by DCausse; author: DCausse):

[schemas/event/primary@master] search: add missing lexeme fields

https://gerrit.wikimedia.org/r/1036566

gerritbot added a project: Patch-For-Review.May 28 2024, 8:50 AM
CodeReviewBot added a comment.May 28 2024, 9:37 AM

dcausse opened https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/708

search: add missing lexeme fields

dcausse merged a task: T365684: Particular lexeme (L1326823) not indexed so search with the Wikidata API returns nothing.May 28 2024, 9:50 AM
dcausse added a comment.May 28 2024, 9:53 AM

The search fields specific to Lexemes are currently ignored causing this NOTICE but also preventing lexemes from being searchable (esp. the new ones).
The schemas should be adapted to support these fields and the lexemes will have to be re-indexed.

gerritbot added a comment.May 28 2024, 9:57 AM

Change #1036585 had a related patch set uploaded (by DCausse; author: DCausse):

[mediawiki/extensions/WikibaseLexemeCirrusSearch@master] Workaround missing lemma fields

https://gerrit.wikimedia.org/r/1036585

CodeReviewBot added a comment.May 28 2024, 1:45 PM

dcausse opened https://gitlab.wikimedia.org/repos/search-platform/cirrus-streaming-updater/-/merge_requests/134

Update test model and log unexpected fields

CodeReviewBot added a comment.May 28 2024, 3:33 PM

ebernhardson merged https://gitlab.wikimedia.org/repos/search-platform/cirrus-streaming-updater/-/merge_requests/134

Update test model and log unexpected fields

gerritbot added a comment.May 28 2024, 3:56 PM

Change #1036585 merged by jenkins-bot:

[mediawiki/extensions/WikibaseLexemeCirrusSearch@master] Workaround missing lemma fields

https://gerrit.wikimedia.org/r/1036585

ReleaseTaggerBot added a project: MW-1.43-notes (1.43.0-wmf.8; 2024-06-04).May 28 2024, 4:00 PM
dcausse mentioned this in rEWLC5e903c77c46b: Workaround missing lemma fields.May 28 2024, 4:48 PM
CodeReviewBot added a comment.May 28 2024, 5:57 PM

dcausse opened https://gitlab.wikimedia.org/repos/search-platform/cirrus-streaming-updater/-/merge_requests/135

Update test model and log unexpected fields

gerritbot added a comment.May 28 2024, 10:05 PM

Change #1036566 merged by jenkins-bot:

[schemas/event/primary@master] search: add missing lexeme fields

https://gerrit.wikimedia.org/r/1036566

CodeReviewBot added a comment.May 28 2024, 10:05 PM

ebernhardson merged https://gitlab.wikimedia.org/repos/search-platform/cirrus-streaming-updater/-/merge_requests/135

Update test model (take 2)

gerritbot added a comment.May 29 2024, 7:41 AM

Change #1036992 had a related patch set uploaded (by DCausse; author: DCausse):

[operations/deployment-charts@master] cirrus-streaming-updater: use latest image

https://gerrit.wikimedia.org/r/1036992

gerritbot added a comment.May 29 2024, 7:52 AM

Change #1036992 merged by jenkins-bot:

[operations/deployment-charts@master] cirrus-streaming-updater: use latest image

https://gerrit.wikimedia.org/r/1036992

dcausse updated the task description. (Show Details)May 29 2024, 10:18 AM
dcausse added a comment.May 29 2024, 10:20 AM

The system should now index lexemes properly.
We still have to reindex all the lexemes to fix the ones created/edited before the fix was applied.

CodeReviewBot added a comment.May 30 2024, 9:58 AM

dcausse merged https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/708

search: add missing lexeme fields

Maintenance_bot removed a project: Patch-For-Review.May 30 2024, 10:31 AM
dcausse moved this task from In Progress to Needs Reporting on the Discovery-Search (Current work) board.Jun 11 2024, 8:36 AM

Triggered a reindex of all the lexemes using https://gitlab.wikimedia.org/repos/search-platform/cirrus-rerender, might take about 3 hours to complete.

dcausse updated the task description. (Show Details)Jun 11 2024, 8:36 AM
Gehel closed this task as Resolved.Jun 14 2024, 7:37 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits