Page MenuHomePhabricator
Create Task
Maniphest T256168

Move beta cluster automatic deployment to a dedicated infrastructure
Open, LowPublic
Actions

Description

CI takes care of updating the Beta cluster infrastructure, that should be moved out of the CI Jenkins to a dedicated infrastructure.

The job lists can be seen at: https://integration.wikimedia.org/ci/view/Beta/

beta-code-update-eqiadTriggers every 10 minutes, update mediawiki code from Gerrit master branches
beta-mediawiki-config-update-eqiadTriggered by CI / Zuul after a change has been merged in operations/mediawiki-config.git
beta-scap-eqiadScap job triggered by the above two jobs
beta-publish-debBuild Scap Debian package and update it on the deployment host
beta-update-databases-eqiadTriggers every hour, runs MediaWiki update.php against each wikis

The jobs are defined using the Jenkins job builder DSL. Our tutorial to set up jenkins job builder and some overview is at https://www.mediawiki.org/wiki/CI/JJB

Upstream doc https://jenkins-job-builder.readthedocs.io/en/latest/

Our DSL files are in integration/config.git under ./jjb/

@hashar guesses those jobs can be moved to a dedicated Jenkins, possibly part of Beta-Cluster-Infrastructure .

Related Objects
Search...

Event Timeline

hashar created this task.Jun 23 2020, 7:32 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 23 2020, 7:32 PM
hashar added a project: good first task.Jun 23 2020, 7:32 PM
hashar added a parent task: T254919: Run two Jenkins master.
taavi claimed this task.Jun 26 2020, 7:00 AM
taavi removed a project: good first task.
taavi subscribed.

@hashar I'm interesting on working on this but I think this would require access to the beta cluster openstack project (which makes me remove good first task).

Restricted Application added a project: User-Majavah. · View Herald TranscriptJun 26 2020, 7:00 AM
hashar added a project: good first task.Jun 26 2020, 8:35 PM

good first task is there for the Release-Engineering-Team WMF team. We have a new person coming in and we created a few basic tasks for the onboarding :-]

Aklapper removed a project: good first task.Jun 27 2020, 12:53 AM

@hashar: good first task are for any contributors and requires that tasks do not require any special permissions in order to test the contribution or to fix the task. Anyone should be able to reproduce.. If this is for onboarding in some team then please either use a dedicated workboard column on your team's workboard for onboarding task, or set the task assignee to the new person coming in, but (in my understanding) please don't set good first task - thanks!

taavi removed taavi as the assignee of this task.Jun 27 2020, 4:45 AM
taavi removed a project: User-Majavah.
thcipriani edited projects, added Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)); removed Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1)).Oct 21 2020, 1:07 AM
thcipriani removed a project: Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)).Apr 20 2021, 12:21 AM
thcipriani added a project: Release-Engineering-Team-TODO (2021-04-01 to 2021-06-30 (Q4)).Apr 20 2021, 12:34 AM
thcipriani removed a project: Release-Engineering-Team (CI & Testing services).Apr 20 2021, 1:09 AM
thcipriani edited projects, added Release-Engineering-Team (Doing); removed Release-Engineering-Team-TODO (2021-04-01 to 2021-06-30 (Q4)).Apr 20 2021, 4:25 AM
hashar mentioned this in T188367: Use cron instead of Jenkins for beta deployments.Jun 4 2021, 12:41 PM
hashar added a parent task: T72597: Jenkins Gearman plugin has deadlock on executor threads (was: Beta Cluster stopped receiving code updates (beta-update-databases-eqiad hung).
thcipriani edited projects, added Quality-and-Test-Engineering-Team; removed Release-Engineering-Team (Doing).Jul 13 2021, 5:42 PM
thcipriani subscribed.

Adding Quality-and-Test-Engineering-Team to determine as product owners if this is still the right path

Jrbranaa moved this task from Inbox to Test Infrastructure on the Quality-and-Test-Engineering-Team board.Sep 7 2021, 4:27 PM
hashar added a project: Jenkins.Feb 8 2023, 1:17 PM
hashar added a subscriber: jnuche.

The releases Jenkins is now deployed with scap using YAML to define the state of the instance (see T323909). From there we should be able to create the config needed to generate and deploy a standalone Jenkins on Beta-Cluster-Infrastructure

Restricted Application added a project: Continuous-Integration-Infrastructure. · View Herald TranscriptFeb 8 2023, 1:17 PM
hashar mentioned this in T183164: Move the beta cluster jobs to a dedicated/standalone Jenkins instance.Feb 8 2023, 1:19 PM
Jdforrester-WMF subscribed.Feb 8 2023, 1:59 PM
TAdeleye_WMF edited projects, added Quality-and-Test-Engineering-Team (Test Infrastructure); removed Quality-and-Test-Engineering-Team.Mar 1 2023, 2:56 PM
thcipriani triaged this task as Low priority.May 1 2023, 10:41 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL