Page MenuHomePhabricator

Automated Tuesday Train via a timer
Open, Needs TriagePublic5 Estimated Story Points

Description

Let's make a timer that automates Tue "stage-train" tasks

Acceptance Criteria

  • Agree on a time for automated deployment every week
  • Patch the deployment calendar to add a new window (along with where to watch for failures)
  • User exists for running timer (T303857)
  • Timer exists on deployment servers
  • Timer does not require any human intervention
  • Timer determines the wmf.X branch/version for the week.
    • The right thing happens if the current week's train task is declined.
  • Timer runs scap stage-train --yes <version> runs to completion without errors under normal conditions.
    • Set TMUX or STY environment variable to something to avoid an error from check_term_multplxr(), or add an option to skip the check for screen/tmux session.
  • Timer can be re-executed if it previously failed.
  • At least during development, the schedule of the timer must be controllable (at least to ~1 minute resolution) by deployers, including the ability to disable it.

Alerting on failure is T310396

Event Timeline

Change 805464 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/puppet@production] Allow deployers to sudo -u mwpresync

https://gerrit.wikimedia.org/r/805464

Change 807972 had a related patch set uploaded (by Hashar; author: Hashar):

[operations/puppet@production] Boilerplate for automatic MediaWiki deployment

https://gerrit.wikimedia.org/r/807972

For the logic behind "Timer determines the wmf.X branch/version for the week" I propose that we let the automated train branch job figure that out (as it does currently) and simply look for the latest wmf/* branch and check that it isn't already in wikiversions.json.

Alternately, the branch cut job could write the branch name and/or version to an artifact (say MW_VERSION) and we could poll https://releases-jenkins.wikimedia.org/job/Automatic%20branch%20cut/lastSuccessfulBuild/artifact/MW_VERSION for changes

Alternately, the branch cut job could write the branch name and/or version to an artifact (say MW_VERSION) and we could poll https://releases-jenkins.wikimedia.org/job/Automatic%20branch%20cut/lastSuccessfulBuild/artifact/MW_VERSION for changes

I like this idea.

Change 809220 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[mediawiki/tools/scap@master] Add auto mode to stage-train

https://gerrit.wikimedia.org/r/809220

Change 809297 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/puppet@production] Setup .gitconfig for mwpresync system user

https://gerrit.wikimedia.org/r/809297

Change 809297 merged by Alexandros Kosiaris:

[operations/puppet@production] Setup .gitconfig for mwpresync system user

https://gerrit.wikimedia.org/r/809297

Change 809712 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/puppet@production] Allow mwbuilder group to access mwdeploy key

https://gerrit.wikimedia.org/r/809712