Page MenuHomePhabricator

[TECH][MSMF] Enable concurrency groups in deployment workflows
Closed, ResolvedPublic3 Estimated Story Points

Description

After an incident involving too many concurrent runs of the "Ship & Deploy" workflow in mismatch finder, we decided to utilize GitHub Actions' concurrency groups.

Docs: https://docs.github.com/en/actions/using-jobs/using-concurrency

Acceptance Criteria:

  • Jobs in the deploy-app workflow are using their own concurrency group
  • Jobs in the deploy-app-staging workflow are using their own concurrency group

Details

Other Assignee
guergana.tzatchkova

Event Timeline

ItamarWMDE renamed this task from [SW][TECH][MSMF] Enable concurrency groups in deployment workflows to [TECH][MSMF] Enable concurrency groups in deployment workflows.Sep 18 2023, 3:36 PM

Task Breakdown notes:

  • How do we verify that this succeeded? Hands on tests (please not on production, staging should be alright).
  • Researching (and possibly documenting) how to verify these kinds of changes should be part of the task.

Tested with manual testing in a staging branch (/staging/**) by pushing an empty commit after the commit with the workflow change:

Proof:

image.png (279×1 px, 36 KB)

Test:
two PRs were created and made to run at the same time.
This is the result:
The run in the /staging/** branch was cancelled: https://github.com/wmde/wikidata-mismatch-finder/actions/runs/6327107547
and the run in /gh-actions/** that was running simultaneously ran until the end. https://github.com/wmde/wikidata-mismatch-finder/actions/runs/6327107547

We need to watch out when we are changing the development branch, this caused a small incident where development branch was accidentally deployed to production (which caused imports to fail due to T313467 already being merged to development). I think we should remove the gh-action/* branch deployment to production, and just rely on local testing and testing on staging, to avoid future issues.