Page MenuHomePhabricator
Create Task
Maniphest T264283

Assess cleaning up Wikibase code around CirrusSearch sanitizer job
Closed, ResolvedPublic
Actions

Description

The sanitizer (or saneitizer) job was disabled during the terms migration; see if we can enable it again, and maybe also improve it / make it more efficient as outlined in T239931 (try unsetting the 'generate-html' flag).

Details

SubjectRepoBranchLines +/-
Override EntityHandler::getParserOutputForIndexingmediawiki/extensions/Wikibasemaster+29 -0
Clarify HTML generation for indexing in ContentHandlermediawiki/coremaster+28 -2
Override getParserOutputForIndexing to ensure HTMLmediawiki/extensions/Jademaster+27 -0
Customize query in gerrit

Related Objects

Event Timeline

Lucas_Werkmeister_WMDE created this task.Oct 1 2020, 8:39 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 1 2020, 8:39 AM
Addshore added a project: Story.Oct 1 2020, 8:40 AM
Lucas_Werkmeister_WMDE updated the task description. (Show Details)Oct 1 2020, 9:03 AM
Lucas_Werkmeister_WMDE moved this task from Backlog to To do on the Wikibase wb_terms leftovers 2020 board.
Lucas_Werkmeister_WMDE claimed this task.Oct 7 2020, 2:56 PM
Lucas_Werkmeister_WMDE moved this task from To do to Doing on the Wikibase wb_terms leftovers 2020 board.

Claiming the task since I started looking into it.

gerritbot added a comment.Oct 7 2020, 4:09 PM

Change 632751 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Jade@master] Override getParserOutputForIndexing to ensure HTML

https://gerrit.wikimedia.org/r/632751

gerritbot added a project: Patch-For-Review.Oct 7 2020, 4:09 PM

Change 632752 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/core@master] Don’t generate HTML in getParserOutputForIndexing

https://gerrit.wikimedia.org/r/632752

Lucas_Werkmeister_WMDE moved this task from Doing to Peer Review on the Wikibase wb_terms leftovers 2020 board.Oct 7 2020, 4:12 PM
gerritbot added a comment.Oct 9 2020, 11:46 AM

Change 633170 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@master] Override EntityHandler::getParserOutputForIndexing

https://gerrit.wikimedia.org/r/633170

gerritbot added a comment.Oct 12 2020, 10:17 AM

Change 632751 abandoned by Lucas Werkmeister (WMDE):
[mediawiki/extensions/Jade@master] Override getParserOutputForIndexing to ensure HTML

Reason:
No longer needed with PS2 of I50f3a530f2, where generating HTML is no longer skipped by default.

https://gerrit.wikimedia.org/r/632751

gerritbot added a comment.Oct 13 2020, 3:24 PM

Change 632752 merged by jenkins-bot:
[mediawiki/core@master] Clarify HTML generation for indexing in ContentHandler

https://gerrit.wikimedia.org/r/632752

ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.14; 2020-10-20).Oct 13 2020, 4:00 PM
gerritbot added a comment.Oct 15 2020, 10:40 AM

Change 633170 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Override EntityHandler::getParserOutputForIndexing

https://gerrit.wikimedia.org/r/633170

Lucas_Werkmeister_WMDE closed this task as Resolved.Oct 15 2020, 2:34 PM
Lucas_Werkmeister_WMDE moved this task from Peer Review to Done on the Wikibase wb_terms leftovers 2020 board.

This task can be closed. We’ve done the assessment, and will turn the sanitizer back on once the above changes are deployed, using T239931: Reduce the impact of the sanitizer on wikidata to track the progress.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits