Page MenuHomePhabricator
Create Task
Maniphest T233584

Re-adjust cirrusSearchLinksUpdate vs cirrusSearchLinksUpdatePrioritized concurrency
Closed, ResolvedPublic
Actions

Description

A recent fix in CirrusSearch caused the cirrusSearchLinksUpdate jobs to be routed to the prio/non-prio topics correctly.
Prior to https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/535869 all jobs were routed to the prioritized topic cirrusSearchLinksUpdate.
Now the cirrusSearchLinksUpdatePrioritized should only be populated for "root" LinksUpdate jobs. Other changes due to template propagation should be scheduled to the cirrusSearchLinksUpdate, the effect is visible on the number of received messages on both of these topics.

We should perhaps re-adjust the concurrency configuration of these topics in changeprop.

Details

SubjectRepoBranchLines +/-
[cirrus] temp disable sanity checkoperations/mediawiki-configmaster+1 -1
CirrusSearchLinksUpdate: Bump concurrency temporarily to 300mediawiki/services/change-propagation/jobqueue-deploymaster+1 -1
[cirrus] increase cirrusSearchLinksUpdate concurrencymediawiki/services/change-propagation/jobqueue-deploymaster+2 -2
Customize query in gerrit

Event Timeline

dcausse created this task.Sep 23 2019, 8:43 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 23 2019, 8:43 AM
dcausse renamed this task from Re-adjust cirrusSearchLinksUpdate vs cirrusSearchLinksUpdate to Re-adjust cirrusSearchLinksUpdate vs cirrusSearchLinksUpdatePrioritized concurrency.Sep 23 2019, 8:53 AM
dcausse added projects: Discovery-Search (Current work), WMF-JobQueue, Services.
dcausse updated the task description. (Show Details)
dcausse moved this task from Incoming to Waiting on the Discovery-Search (Current work) board.
gerritbot added a comment.Sep 24 2019, 8:20 AM

Change 538835 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/services/change-propagation/jobqueue-deploy@master] [cirrus] increase cirrusSearchLinksUpdate concurrency

https://gerrit.wikimedia.org/r/538835

gerritbot added a project: Patch-For-Review.Sep 24 2019, 8:20 AM
• mobrovac edited projects, added Services (doing), Platform Team Workboards (Clinic Duty Team); removed Services.Sep 24 2019, 10:35 AM
gerritbot added a comment.Sep 24 2019, 10:37 AM

Change 538835 merged by Mobrovac:
[mediawiki/services/change-propagation/jobqueue-deploy@master] [cirrus] increase cirrusSearchLinksUpdate concurrency

https://gerrit.wikimedia.org/r/538835

gerritbot added a comment.Sep 24 2019, 10:39 AM

Change 538854 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/change-propagation/jobqueue-deploy@master] CirrusSearchLinksUpdate: Bump concurrency temporarily to 300

https://gerrit.wikimedia.org/r/538854

• mobrovac claimed this task.Sep 24 2019, 10:41 AM
• mobrovac triaged this task as High priority.
• mobrovac moved this task from Inbox to Doing(WIP:5) on the Platform Team Workboards (Clinic Duty Team) board.
gerritbot added a comment.Sep 24 2019, 10:53 AM

Change 538854 merged by Mobrovac:
[mediawiki/services/change-propagation/jobqueue-deploy@master] CirrusSearchLinksUpdate: Bump concurrency temporarily to 300

https://gerrit.wikimedia.org/r/538854

Stashbot added a comment.Sep 24 2019, 10:55 AM

Mentioned in SAL (#wikimedia-operations) [2019-09-24T10:55:05Z] <mobrovac@deploy1001> Started deploy [cpjobqueue/deploy@7857639]: Bump CirrusSearchLinksUpdate concurrency to clear the queue - T233584

Stashbot added a comment.Sep 24 2019, 10:56 AM

Mentioned in SAL (#wikimedia-operations) [2019-09-24T10:56:04Z] <mobrovac@deploy1001> Finished deploy [cpjobqueue/deploy@7857639]: Bump CirrusSearchLinksUpdate concurrency to clear the queue - T233584 (duration: 01m 00s)

Maintenance_bot removed a project: Patch-For-Review.Sep 24 2019, 11:10 AM
gerritbot added a comment.Sep 24 2019, 12:11 PM

Change 538872 had a related patch set uploaded (by DCausse; owner: DCausse):
[operations/mediawiki-config@master] [cirrus] temp disable sanity check

https://gerrit.wikimedia.org/r/538872

gerritbot added a project: Patch-For-Review.Sep 24 2019, 12:11 PM
gerritbot added a comment.Sep 24 2019, 12:15 PM

Change 538872 merged by jenkins-bot:
[operations/mediawiki-config@master] [cirrus] temp disable sanity check

https://gerrit.wikimedia.org/r/538872

Stashbot added a comment.Sep 24 2019, 12:20 PM

Mentioned in SAL (#wikimedia-operations) [2019-09-24T12:20:30Z] <dcausse@deploy1001> Synchronized wmf-config/CirrusSearch-production.php: T233584 [cirrus] temp disable sanity check (duration: 00m 55s)

Maintenance_bot removed a project: Patch-For-Review.Sep 24 2019, 1:10 PM
Stashbot added a comment.Sep 25 2019, 7:51 AM

Mentioned in SAL (#wikimedia-operations) [2019-09-25T07:51:20Z] <dcausse@deploy1001> Synchronized wmf-config/CirrusSearch-production.php: T233584 revert: [cirrus] temp disable sanity check (duration: 01m 05s)

• mobrovac closed this task as Resolved.Oct 7 2019, 11:15 AM
• mobrovac moved this task from Doing(WIP:5) to Done on the Platform Team Workboards (Clinic Duty Team) board.
• mobrovac edited projects, added Services (done); removed Services (doing).

The jobs are stable even with the higher concurrency, so I'd be inclined of keeping the current level. Resolving.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL