Page MenuHomePhabricator
Create Task
Maniphest T349838

[Search Update Pipeline] Create/adapt index comparing script for checking correctness
Closed, ResolvedPublic3 Estimated Story Points
Actions

Description

To make sure the search update pipeline yields expected ES updates, we need a script that compares the contents of two indices. There already is such script, compare-clusters.py but this is limited to checking page IDs.

AC

  • There is (an updated version of the existing) script that compares at least
    • page ID
    • revision ID
    • title

Details

SubjectRepoBranchLines +/-
Adapt compare-clusters.py to compare existence, revision, and titlemediawiki/extensions/CirrusSearchmaster+110 -58
Adapt compare-clusters.py, utilize python 3.x featuresmediawiki/extensions/CirrusSearchmaster+111 -58
Customize query in gerrit

Event Timeline

pfischer created this task.Oct 26 2023, 3:59 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 26 2023, 3:59 PM
Gehel moved this task from needs triage to Current work on the Discovery-Search board.Oct 30 2023, 4:15 PM
Gehel edited projects, added Discovery-Search (Current work); removed Discovery-Search.
Gehel set the point value for this task to 3.Oct 30 2023, 4:32 PM
Gehel moved this task from Incoming to Ready for Dev -- SWE on the Discovery-Search (Current work) board.
EBernhardson updated the task description. (Show Details)Oct 30 2023, 4:33 PM
pfischer claimed this task.Nov 1 2023, 9:32 AM
pfischer moved this task from Ready for Dev -- SWE to In Progress on the Discovery-Search (Current work) board.
gerritbot added a comment.Nov 7 2023, 4:28 PM

Change 972413 had a related patch set uploaded (by Peter Fischer; author: Peter Fischer):

[mediawiki/extensions/CirrusSearch@master] Adapt compare-clusters.py to compare existence, revision, and title

https://gerrit.wikimedia.org/r/972413

gerritbot added a project: Patch-For-Review.Nov 7 2023, 4:28 PM
pfischer moved this task from In Progress to Needs review on the Discovery-Search (Current work) board.Nov 7 2023, 4:39 PM
gerritbot added a comment.Nov 15 2023, 8:48 PM

Change 972413 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Adapt compare-clusters.py to compare existence, revision, and title

https://gerrit.wikimedia.org/r/972413

ReleaseTaggerBot added a project: MW-1.42-notes (1.42.0-wmf.7; 2023-11-28).Nov 15 2023, 9:00 PM
Maintenance_bot removed a project: Patch-For-Review.Nov 15 2023, 9:10 PM
gerritbot added a comment.Nov 16 2023, 3:41 PM

Change 975010 had a related patch set uploaded (by Peter Fischer; author: Peter Fischer):

[mediawiki/extensions/CirrusSearch@master] Adapt compare-clusters.py, utilize python 3.x features

https://gerrit.wikimedia.org/r/975010

gerritbot added a project: Patch-For-Review.Nov 16 2023, 3:41 PM
gerritbot added a comment.Nov 16 2023, 3:45 PM

Change 975010 abandoned by Peter Fischer:

[mediawiki/extensions/CirrusSearch@master] Adapt compare-clusters.py, utilize python 3.x features

Reason:

https://gerrit.wikimedia.org/r/975010

Maintenance_bot removed a project: Patch-For-Review.Nov 16 2023, 4:10 PM
EBernhardson moved this task from Needs review to To Be Deployed on the Discovery-Search (Current work) board.Nov 16 2023, 6:08 PM
EBernhardson moved this task from To Be Deployed to Needs Reporting on the Discovery-Search (Current work) board.Nov 29 2023, 8:23 PM
Gehel closed this task as Resolved.Dec 1 2023, 9:21 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL