Allow batch_size to be configured for each deploy stage
Closed, ResolvedPublic

Description

Building off of the staged deploy implemented in T109861: Scap3 should break up remote deploy tasks, we should allow the batch size to be configurable for each stage.

Per our sprint meeting discussion last Friday, the main reasoning behind breaking up deployment into distinct stages is to be able to independently control the batch sizes, ultimately leading to a more efficient overall process. Specifically, Git fetching/cloning and possibly other preparatory/build tasks could be executed in larger batches concurrently, while the final promotion (symlink swaps, service restart, etc.) are still done is a more serial fashion (rolling restart).

dduvall created this task.Sep 17 2015, 12:27 AM
dduvall claimed this task.
dduvall added a project: Scap.
dduvall added a subscriber: dduvall.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 17 2015, 12:27 AM
dduvall triaged this task as "Low" priority.Sep 17 2015, 12:28 AM
dduvall set Security to None.
dduvall moved this task from Needs triage to Services MVP on the Scap board.

Change 239016 had a related patch set uploaded (by Dduvall):
Support batch size configuration per stage

https://gerrit.wikimedia.org/r/239016

greg added a subscriber: greg.Sep 19 2015, 1:41 AM

Question: For the Service's use case (T102667: Create or improve the RESTBase deploy method), they have "should allow batches (specify via config or at runtime)".

Does this implement that request, or is that request more asking for what we do with the train ("deploy to this many users/wikis for now, wait, then at some arbitrary later time deploy to more")?

Change 239016 merged by jenkins-bot:
Support batch size configuration per stage

https://gerrit.wikimedia.org/r/239016

Does this implement that request,

Yes. There are some services, like Parsoid, which have large cluster instances, so doing server-by-server deploys (and restarts) would take ages. Instead, the idea is to execute the same instructions on a subset of hosts in parallel.

greg added a comment.Sep 21 2015, 10:35 PM

Sweet, thanks for the confirmation, @mobrovac.

dduvall closed this task as "Resolved".Sep 23 2015, 5:30 PM
dduvall moved this task from Services MVP to Done on the Scap board.