Page MenuHomePhabricator

Depool and repool SCB services during deploys
Closed, ResolvedPublic0 Story Points

Description

We can now depool servers when we deploy to them and repool them back once the service has been restarted. We should tell Scap3 to do it for each SCB service so as to keep the amount of failed requests due to deploys to a minimum.

Details

Related Gerrit Patches:
mediawiki/services/eventstreams/deploy : masterScap: Depool and repool EventStreams during restarts
mediawiki/services/trending-edits/deploy : masterScap: Depool and repool Trending Edits during restarts
mediawiki/services/mobileapps/deploy : masterScap: Depool and repool MobileApps during restarts
mediawiki/services/electron-render/deploy : masterScap: Depool and repool PDFRender during restarts
mediawiki/services/graphoid/deploy : masterScap: Depool and repool Graphoid during restarts
mediawiki/services/cxserver/deploy : masterScap: Depool and repool CXServer during restarts
mediawiki/services/citoid/deploy : masterScap: Depool and repool Citoid during restarts
mediawiki/services/mathoid/deploy : masterScap: Depool and repool Mathoid during restarts

Event Timeline

mobrovac created this task.Sep 2 2016, 2:42 PM
Restricted Application added a project: VisualEditor. · View Herald TranscriptSep 2 2016, 2:42 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
mobrovac triaged this task as Medium priority.Sep 2 2016, 2:42 PM
Mvolz moved this task from Backlog to Production on the Citoid board.Oct 10 2016, 2:23 PM
Restricted Application added a project: ContentTranslation. · View Herald TranscriptOct 10 2016, 2:23 PM
Amire80 added a subscriber: Amire80.

Mmmm, this is super-internal and technical, but it's on the ContentTranslation board, so I have to triage it :)

When is this supposed to happen and who will do it?

demon moved this task from Needs triage to Services improvements on the Scap board.Dec 8 2016, 1:57 AM
mobrovac edited projects, added Services (next); removed Services.Dec 16 2016, 9:57 PM

Change 329476 had a related patch set uploaded (by Mobrovac):
Scap: Depool and repool Mathoid during restarts

https://gerrit.wikimedia.org/r/329476

Change 329476 merged by Mobrovac:
Scap: Depool and repool Mathoid during restarts

https://gerrit.wikimedia.org/r/329476

Mentioned in SAL (#wikimedia-operations) [2016-12-28T13:19:08Z] <mobrovac@tin> Starting deploy [mathoid/deploy@79fdd56]: Add depooling and repooling to Mathoid T144602

Mentioned in SAL (#wikimedia-operations) [2016-12-28T13:21:39Z] <mobrovac@tin> Finished deploy [mathoid/deploy@79fdd56]: Add depooling and repooling to Mathoid T144602 (duration: 02m 31s)

Change 329477 had a related patch set uploaded (by Mobrovac):
Scap: Depool and repool Citoid during restarts

https://gerrit.wikimedia.org/r/329477

Change 329477 merged by Mobrovac:
Scap: Depool and repool Citoid during restarts

https://gerrit.wikimedia.org/r/329477

Mentioned in SAL (#wikimedia-operations) [2016-12-28T13:31:02Z] <mobrovac@tin> Starting deploy [citoid/deploy@da96f4b]: Add depooling and repooling to Citoid T144602

Mentioned in SAL (#wikimedia-operations) [2016-12-28T13:33:01Z] <mobrovac@tin> Finished deploy [citoid/deploy@da96f4b]: Add depooling and repooling to Citoid T144602 (duration: 01m 59s)

Change 329479 had a related patch set uploaded (by Mobrovac):
Scap: Depool and repool CXServer during restarts

https://gerrit.wikimedia.org/r/329479

Change 329479 merged by Mobrovac:
Scap: Depool and repool CXServer during restarts

https://gerrit.wikimedia.org/r/329479

Mentioned in SAL (#wikimedia-operations) [2016-12-28T13:41:18Z] <mobrovac@tin> Starting deploy [cxserver/deploy@0279029]: Add depooling and repooling to CXServer T144602

Mentioned in SAL (#wikimedia-operations) [2016-12-28T13:44:41Z] <mobrovac@tin> Finished deploy [cxserver/deploy@0279029]: Add depooling and repooling to CXServer T144602 (duration: 03m 22s)

Change 329480 had a related patch set uploaded (by Mobrovac):
Scap: Depool and repool Graphoid during restarts

https://gerrit.wikimedia.org/r/329480

Change 329480 merged by Mobrovac:
Scap: Depool and repool Graphoid during restarts

https://gerrit.wikimedia.org/r/329480

Mentioned in SAL (#wikimedia-operations) [2016-12-28T13:49:36Z] <mobrovac@tin> Starting deploy [graphoid/deploy@151f26c]: Add depooling and repooling to Graphoid T144602

Mentioned in SAL (#wikimedia-operations) [2016-12-28T13:51:14Z] <mobrovac@tin> Finished deploy [graphoid/deploy@151f26c]: Add depooling and repooling to Graphoid T144602 (duration: 01m 37s)

Change 329481 had a related patch set uploaded (by Mobrovac):
Scap: Depool and repool PDFRender during restarts

https://gerrit.wikimedia.org/r/329481

Change 329481 merged by Mobrovac:
Scap: Depool and repool PDFRender during restarts

https://gerrit.wikimedia.org/r/329481

Mentioned in SAL (#wikimedia-operations) [2016-12-28T13:59:05Z] <mobrovac@tin> Starting deploy [electron-render/deploy@b2a820e]: Add depooling and repooling to PDFRender T144602

Mentioned in SAL (#wikimedia-operations) [2016-12-28T14:00:31Z] <mobrovac@tin> Finished deploy [electron-render/deploy@b2a820e]: Add depooling and repooling to PDFRender T144602 (duration: 01m 26s)

Change 329482 had a related patch set uploaded (by Mobrovac):
Scap: Depool and repool MobileApps during restarts

https://gerrit.wikimedia.org/r/329482

Change 329482 merged by Mobrovac:
Scap: Depool and repool MobileApps during restarts

https://gerrit.wikimedia.org/r/329482

Mentioned in SAL (#wikimedia-operations) [2016-12-28T14:05:08Z] <mobrovac@tin> Starting deploy [mobileapps/deploy@ae22656]: Add depooling and repooling to MCS T144602

Mentioned in SAL (#wikimedia-operations) [2016-12-28T14:07:41Z] <mobrovac@tin> Finished deploy [mobileapps/deploy@ae22656]: Add depooling and repooling to MCS T144602 (duration: 02m 34s)

Change 329484 had a related patch set uploaded (by Mobrovac):
Scap: Depool and repool Trending Edits during restarts

https://gerrit.wikimedia.org/r/329484

Change 329484 merged by Mobrovac:
Scap: Depool and repool Trending Edits during restarts

https://gerrit.wikimedia.org/r/329484

Mentioned in SAL (#wikimedia-operations) [2016-12-28T14:11:09Z] <mobrovac@tin> Starting deploy [trending-edits/deploy@c5d239b]: Add depooling and repooling to Trending Edits T144602

Mentioned in SAL (#wikimedia-operations) [2016-12-28T14:12:30Z] <mobrovac@tin> Finished deploy [trending-edits/deploy@c5d239b]: Add depooling and repooling to Trending Edits T144602 (duration: 01m 21s)

Change 329488 had a related patch set uploaded (by Mobrovac):
Scap: Depool and repool EventStreams during restarts

https://gerrit.wikimedia.org/r/329488

Change 329488 merged by Mobrovac:
Scap: Depool and repool EventStreams during restarts

https://gerrit.wikimedia.org/r/329488

mobrovac closed this task as Resolved.Dec 28 2016, 2:25 PM
mobrovac edited projects, added Services (done); removed Services (doing), Patch-For-Review.

Completed. All of the SCB services are now automatically depooled / repooled during deployments. Resolving.

greg awarded a token.Jan 3 2017, 9:19 PM
greg added a subscriber: greg.

\o/

Jdforrester-WMF set the point value for this task to 0.Jan 10 2017, 11:25 PM