Page MenuHomePhabricator
Create Task
Maniphest T279322

Design m8s deployment workflows and tooling
Closed, ResolvedPublic
Actions

Description

Now that we have jobs capable of building deployable images complete with multiple versions of MediaWiki, configuration, security patches, private settings, we need to deploy them to both k8s and legacy servers. Let's sketch out what that could look like from a deployer's perspective.

We (@dancy, @jeena, and @dduvall) brainstormed during our meeting today and came up with the following. It focuses on backport deployments. We also need to define commands/workflows for security patch updates, private settings updates, and train.

deployment commands

scap backport [--list]
scap backport change_url [change_url]
scap rollback
scap backport --revert change_url [change_url]

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedakosiarisT198901 Migrate production services to kubernetes using the pipeline
 ResolvedClement_GoubertT238770 Deploy MediaWiki to Wikimedia production in containers
ResolvedNoneT279322 Design m8s deployment workflows and tooling
 ResolvedjeenaT287041 Scap Backport --list command
 ResolvedNoneT287042 Scap backport change_url command
 ResolvedNoneT287993 Train-dev: Ability to deploy to k8s
 ResolvedjeenaT294453 Scap backport change_url: Validate that the specified changes are suitable
ResolveddduvallT294454 Scap backport change_url: approve changes
 DeclinedNoneT294455 Scap backport change_url: update values.yaml
 InvalidNoneT294457 Scap backport change_url: copy files for legacy deployment
 ResolveddduvallT294459 Scap (backport) should support configuration of Gerrit URL
 DeclinedNoneT294462 Scap backport change_url: deploy to k8s
 ResolvedjeenaT294466 Scap backport change_url: legacy deployment
 ResolvedjeenaT295494 Scap backport change_url: Implement “wait for my commits to be merged”
 ResolvedjeenaT295495 Scap backport change_url: Implement “what are the commit hashes from the two repos that correspond to my merged commit(s)”.
 ResolvedjeenaT303755 Scap backport change_url: make gerrit plugin authorize with ssh
 ResolvedNoneT303967 Display mergeable status in scap backport --list output
 ResolvedjeenaT308474 scap backport change_url: Update to use new zuul plugin
 ResolvedjeenaT308476 scap backport change_url: Deploy to mwdebug
 ResolveddduvallT287045 Scap rollback command
 ResolvedjeenaT287046 scap backport --revert command

Event Timeline

dduvall created this task.Apr 5 2021, 6:57 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 5 2021, 6:57 PM
dduvall mentioned this in T279324: Provide a development environment for testing m8s deployment tooling.Apr 5 2021, 7:15 PM
dduvall mentioned this in T279340: Include MediaWiki testing in .pipeline/config.yaml execution graph.Apr 5 2021, 7:21 PM
dduvall updated the task description. (Show Details)Apr 5 2021, 7:30 PM
thcipriani subscribed.Apr 5 2021, 7:31 PM

scap backport --revert change_id [change_id]

One quick point here, a change_id can refer to more than 1 change: https://gerrit.wikimedia.org/r/q/If04177f2a9f681002be5eafd97cbc5800aed2e27

I'm not sure if that's a problem or not, but something to be aware of in the design

• mmodell awarded a token.Apr 5 2021, 7:39 PM
thcipriani mentioned this in T193311: Track scap syncs that are part of a given SWAT window .Apr 6 2021, 4:44 PM
dduvall updated the task description. (Show Details)Apr 6 2021, 4:46 PM
dduvall added a comment.Apr 6 2021, 4:48 PM
In T279322#6973652, @thcipriani wrote:

scap backport --revert change_id [change_id]

One quick point here, a change_id can refer to more than 1 change: https://gerrit.wikimedia.org/r/q/If04177f2a9f681002be5eafd97cbc5800aed2e27

I'm not sure if that's a problem or not, but something to be aware of in the design

I've changed the spec to take change_url as a parameter which could be either a change number or a full url (the contains the number), the important thing being that we should reference change numbers which are unique and not IDs which, as you mentioned, are not.

taavi subscribed.Apr 6 2021, 5:02 PM
akosiaris subscribed.Apr 7 2021, 3:05 PM
thcipriani edited projects, added Release-Engineering-Team (Doing); removed Release-Engineering-Team-TODO (2021-04-01 to 2021-06-30 (Q4)), Release-Engineering-Team.Apr 20 2021, 4:25 AM
thcipriani mentioned this in T278988: Automate MediaWiki wmf branch image builds.Apr 20 2021, 4:44 PM
gerritbot added a comment.Jun 25 2021, 10:21 PM

Change 701615 had a related patch set uploaded (by Jeena Huneidi; author: Jeena Huneidi):

[mediawiki/tools/scap@master] [WIP] scap backport command

https://gerrit.wikimedia.org/r/701615

gerritbot added a project: Patch-For-Review.Jun 25 2021, 10:21 PM
• brennen added a project: User-brennen.Jun 29 2021, 10:19 PM
• brennen subscribed.
• brennen moved this task from Backlog to Radar on the User-brennen board.Jun 30 2021, 3:06 PM
dduvall assigned this task to jeena.Jul 12 2021, 6:36 PM
dduvall triaged this task as Medium priority.

@jeena assigning this to reflect that you have a WIP patch for scap backport --list.

jijiki subscribed.Jul 19 2021, 7:17 PM
jeena updated the task description. (Show Details)Jul 20 2021, 8:16 PM
jeena removed jeena as the assignee of this task.Jul 20 2021, 8:24 PM
jeena updated the task description. (Show Details)
jeena closed subtask T287041: Scap Backport --list command as Resolved.Jul 27 2021, 10:52 PM
jeena added a comment.Aug 17 2021, 6:43 PM

Related task: T287570

thcipriani mentioned this in T287570: Ensure the code is deployed to mediawiki on k8s when it is deployed to production.Aug 23 2021, 4:28 PM
ArielGlenn subscribed.Aug 25 2021, 7:55 AM
jeena closed subtask T287045: Scap rollback command as Declined.Jun 13 2022, 4:04 PM
jeena mentioned this in T287045: Scap rollback command.
thcipriani reopened subtask T287045: Scap rollback command as Open.Jun 13 2022, 4:43 PM
dduvall changed the status of subtask T287045: Scap rollback command from Open to In Progress.Jun 14 2022, 4:05 PM
dduvall changed the status of subtask T287045: Scap rollback command from In Progress to Open.Jun 14 2022, 11:26 PM
dduvall closed subtask T287045: Scap rollback command as Resolved.Jun 27 2022, 5:08 PM
dancy closed subtask T287046: scap backport --revert command as Resolved.Jun 29 2022, 5:45 PM
dancy reopened subtask T287046: scap backport --revert command as Open.Sep 2 2022, 7:24 PM
thcipriani edited projects, added Release-Engineering-Team (Priority Backlog 📥); removed Release-Engineering-Team (Doing).Sep 7 2022, 3:51 PM
dancy closed subtask T287042: Scap backport change_url command as Resolved.Sep 7 2022, 4:22 PM
jeena closed subtask T287046: scap backport --revert command as Resolved.Sep 9 2022, 8:46 PM
thcipriani closed this task as Resolved.Jun 9 2023, 9:19 PM

Looks to me like the scope of this task is complete:

  • scap backport is a thing.
  • We're deploying images to k8s.

Nice work @dancy @jeena and @dduvall

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits