Page MenuHomePhabricator

EPIC: Migrate jobs from Zuulv2 to Zuulv3+
Open, MediumPublic

Description

As an exploratory phase, lets document the various kind of jobs we have in Jenkins. That will help identify the one to migrate to a new system, the ones to drop and the ones to actually migrate to the new Zuul.

As of January 7 we had 824 jobs in the CI Jenkins:

./jenkins-jobs --conf jenkins_jobs.ini list > /dev/null
INFO:root:Matching jobs: 824

  • Matrix projects:
    • -php-security: T309655.
    • operations-puppet-catalog-compiler
  • 140 PipelineLib jobs (and their 140 trigger-* jobs) - TODO FILE A TASK
  • Debian Glue jobs - Note: SRE started migrating repositories to GitLab using their own system TODO FILE A TASK
  • selenium-daily-beta-*

Jobs publishing to doc.wikimedia.org:

  • *-publish
  • *-phpmetrics
  • Jobs that are also triggered by timer/polling:
    • mediawiki-core-doxygen
    • mwcore-phpunit-coverage-master
    • operations-puppet-doc

Jobs on a timer:

  • quibble-daily-Cognate-vendor-mysql-php74
  • quibble-daily-InterwikiSorting-vendor-mysql-php74
  • mobileapps-periodic-test
  • mediawiki-i18n-check
  • The four Code health which interacts with SonarQube

With ssh-agent, should probably move to release Jenkins instead:

  • train-deploy-notes
  • wikidata-query-builder-build
  • wikidata-query-gui-build
  • wikimedia-portals-build
  • Maven releases jobs and analytics-refinery-update-jars and their nasty setfacl
  • wikibase-daily-npm-audit-daily-node20-npmaudit - T383134
  • fresh-test requires Docker. We need Nodepool to provide a standalone Docker (currently we use Qemu)

Internal jobs that do not need to be migrated?

  • castor-save-workspace-cache
  • fail-archived-repositories
  • integration-config-qa
  • publish-to-doc

Tightly tied to SRE managed Debian packages:

  • operations-dns-lint
  • helm-lint
  • operations-puppet-logstash-filter-verifier
  • operations-puppet-tests-bullseye
  • The fundraising jobs
    • The different branching model. See T328240#8639790 for the problem statement and a solution (align repos branches):
      • quibble-donationinterface-REL1_39-php74
      • quibble-fundraising-cldr-REL1_39-php74
      • quibble-fundraising-FundraisingEmailUnsubscribe-REL1_39-php74
    • wikimedia-fundraising-civicrm-bullseye

Puppet compiler. Maybe they could use their own Jenkins:

  • operations-puppet-catalog-compiler-test
  • operations-puppet-catalog-compiler-puppet7-test

Event Timeline

hashar changed the task status from Open to In Progress.Jan 7 2025, 10:28 PM
thcipriani subscribed.

This list of job types looks reasonably complete to me, which would mean this task is complete. But this task is somewhat of an epic insofar as it's growing subtasks and contains a nice list of things needing to be migrated.

I'm going to re-title the task and make it an epic tracking task.

thcipriani renamed this task from Identify the different classes of jobs to EPIC: Migrate jobs from Zuulv2 to Zuulv3+.Jun 6 2025, 6:20 PM
thcipriani changed the task status from In Progress to Open.
thcipriani triaged this task as Medium priority.
thcipriani edited projects, added: Epic; removed: Essential-Work.