Separate off ChangePropagation for JobQueue as a new deployment
Closed, ResolvedPublic

Description

As part of the effort to move the JobQueue transport to the EventBus sub-system, we have to create a separate deployment repository for the ChangePropagation instance that is going to be handling the jobs. This will allow us to (i) easily move the processing elsewhere if resources start being a problem; and (ii) separate the processing of jobs from the processing of asynchronous updates.

This instance should initially be placed on SCB. The deploy repo is change-propagation/jobqueue-deploy.

mobrovac created this task.Thu, Sep 7, 2:44 PM
Restricted Application edited projects, added Services, Analytics; removed Services (doing). · View Herald TranscriptThu, Sep 7, 2:44 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 376527 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/cp-jobqueue@master] Initial add of the Scap3 configuration

https://gerrit.wikimedia.org/r/376527

mobrovac updated the task description. (Show Details)Thu, Sep 7, 2:49 PM

Change 376527 merged by Mobrovac:
[mediawiki/services/cp-jobqueue@master] Initial add of the Scap3 configuration

https://gerrit.wikimedia.org/r/376527

Change 376707 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[operations/puppet@production] Add the cp-jobqueue profile

https://gerrit.wikimedia.org/r/376707

Change 376707 merged by Alexandros Kosiaris:
[operations/puppet@production] Add the cp-jobqueue profile

https://gerrit.wikimedia.org/r/376707

Change 376724 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[operations/puppet@production] ChangeProp-JobQueue: Fix the repo location and name

https://gerrit.wikimedia.org/r/376724

Change 376724 merged by Alexandros Kosiaris:
[operations/puppet@production] ChangeProp-JobQueue: Fix the repo location and name

https://gerrit.wikimedia.org/r/376724

Change 376730 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/change-propagation/jobqueue-deploy@master] Move the repo over from cp-jobqueue

https://gerrit.wikimedia.org/r/376730

Change 376730 merged by Mobrovac:
[mediawiki/services/change-propagation/jobqueue-deploy@master] Move the repo over from cp-jobqueue

https://gerrit.wikimedia.org/r/376730

mobrovac updated the task description. (Show Details)Fri, Sep 8, 2:50 PM

The repo has been set up and cloned on tin and the ops/puppet profile created and merged. Left to do is to add the profile to SCB's role, which is scheduled to happen on Monday, 2017-09-08.

Change 377213 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[operations/puppet@production] CP-JobQueue: Add the service to SCB

https://gerrit.wikimedia.org/r/377213

Change 377213 merged by Giuseppe Lavagetto:
[operations/puppet@production] CP-JobQueue: Add the service to SCB

https://gerrit.wikimedia.org/r/377213

Change 377221 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/change-propagation/jobqueue-deploy@master] Bug fix: The service's name is cpjobqueue, not cp-jobqueue

https://gerrit.wikimedia.org/r/377221

Change 377221 merged by Mobrovac:
[mediawiki/services/change-propagation/jobqueue-deploy@master] Bug fix: The service's name is cpjobqueue, not cp-jobqueue

https://gerrit.wikimedia.org/r/377221

Mentioned in SAL (#wikimedia-operations) [2017-09-11T09:39:34Z] <mobrovac@tin> Started deploy [cpjobqueue/deploy@e73a10f]: Initial deploy of cpjobqueue - T175281

Mentioned in SAL (#wikimedia-operations) [2017-09-11T09:39:55Z] <mobrovac@tin> Finished deploy [cpjobqueue/deploy@e73a10f]: Initial deploy of cpjobqueue - T175281 (duration: 00m 20s)

mobrovac closed this task as Resolved.

Everything is set up now, and the cpjobqueue service is live in production on the SCB cluster (currently idling pending the resolution of T175210: Select candidate jobs for transferring to the new infrastucture). Calling this done!