Page MenuHomePhabricator
Create Task
Maniphest T233587

Optimize the Sanitizer to produce a single job if all elasticsearch clusters are to be updated
Closed, ResolvedPublic
Actions

Description

Currently if the sanitizer detects a discrepancy on all the checked clusters it will produce as many messages as we have clusters. We can optimize this process by sending a single message with the cluster param set to null. This should help reduce the number of messages processed on the CirrusSearchLinksUpdate topic.

Details

SubjectRepoBranchLines +/-
Optimize the CheckerJob process by trying to send a single job for all clustersmediawiki/extensions/CirrusSearchmaster+653 -14
Customize query in gerrit

Related Objects

Event Timeline

dcausse created this task.Sep 23 2019, 8:59 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 23 2019, 8:59 AM
dcausse renamed this task from Opti to Optimize the Sanitizer to produce a single job if all elasticsearch clusters are to be updated.Sep 23 2019, 9:02 AM
dcausse triaged this task as Medium priority.
dcausse added projects: Discovery-Search (Current work), CirrusSearch.
dcausse updated the task description. (Show Details)
Restricted Application edited projects, added Discovery-Search; removed Discovery-Search (Current work). · View Herald TranscriptSep 23 2019, 9:02 AM
dcausse claimed this task.Oct 1 2019, 5:37 PM
dcausse moved this task from needs triage to Current work on the Discovery-Search board.
dcausse edited projects, added Discovery-Search (Current work); removed Discovery-Search.
gerritbot added a comment.Oct 2 2019, 2:58 PM

Change 540418 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/CirrusSearch@master] Optimize the CheckerJob process by trying to send a single job for all clusters

https://gerrit.wikimedia.org/r/540418

gerritbot added a project: Patch-For-Review.Oct 2 2019, 2:58 PM
dcausse moved this task from Incoming to Needs review on the Discovery-Search (Current work) board.Oct 7 2019, 10:05 AM
gerritbot added a comment.Oct 8 2019, 9:08 PM

Change 540418 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Optimize the CheckerJob process by trying to send a single job for all clusters

https://gerrit.wikimedia.org/r/540418

Maintenance_bot removed a project: Patch-For-Review.Oct 8 2019, 9:11 PM
ReleaseTaggerBot added a project: MW-1.35-notes (1.35.0-wmf.2; 2019-10-15).Oct 8 2019, 10:00 PM
dcausse closed this task as Resolved.Oct 9 2019, 2:21 PM
dcausse moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.
dcausse mentioned this in rECIR5068b6b2f032: Optimize the CheckerJob process by trying to send a single job for all clusters.Oct 9 2019, 10:12 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL