Page MenuHomePhabricator

[FY24-25 WE6.2.1] Publish pre-train single version containers
Closed, ResolvedPublicGoal

Description

hypothesis
If we publish a versioned build of MediaWiki, extensions, skins, and Wikimedia configuration at least once per day we will uncover new constraints and establish a baseline of wallclock time needed to perform a build.

Design and implement a process that will produce a Open Container Initiative (OCI) container image including MediaWiki core with Wikimedia project specific extensions, skins, and configuration suitable for running in pre-train validation environments. An ideal process should eventually provide artifacts useful for local use, beta cluster use, and group -1 production use. Local dev and beta cluster accessible artifacts must not include embargoed security patches or other Wikimedia production secrets.

Details

Due Date
Sep 30 2024, 11:59 PM
Related Changes in Gerrit:
Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
mwscript: Containerize MediaWiki script executionrepos/releng/scap!500dduvallreview/vertvert-9a75master
mwscript: Containerize MediaWiki script executionrepos/releng/scap!478dduvallreview/redo-mwscript-docker-0744master
build-images: Skip JSON/MD5 l10n generationrepos/releng/scap!461dduvallmaster-I9bb0d925694f4aef718a57358579932e7539465bmaster
jobs: Clean up l10n files using docker and copy all private settingsrepos/releng/jenkins-deploy!84dduvallmaster-Iaa27d933a143c2183ac96008e76f985db79a7f30master
deploy: Support docker based mwscript executionrepos/releng/train-dev!89dduvallmain-I85c9dd768f263f5fd082a3a48a9b9367fdd13a94main
mwscript: Containerize MediaWiki script executionrepos/releng/scap!450dduvallmaster-I79d2e648a0df8fb15218ab5d601e94e603bcf6cfmaster
jobs: Refactor `publishMWSingleVersion` to use `scap build-images`repos/releng/jenkins-deploy!83dduvallmaster-Iaa27d933a143c2183ac96008e76f985db79a7f30master
kubernetes: Provide `build-images` subcommandrepos/releng/scap!436dduvallmaster-Ibc0c588214b3db506c965ac3444473699e715a29master
jobs: Escape `${` in `publishMWSingleVersion` job scriptrepos/releng/jenkins-deploy!81dduvallmaster-Ib608dfd7f6269664c36487dc1ec09e9007048b1fmaster
jobs: Fix use of relativeTargetDirectoryrepos/releng/jenkins-deploy!80dduvallmaster-I8f38d33fb6cac8d6c0321fe29fec5ec03aa67199master
jobs: Define a job that publishes a single version MediaWiki imagerepos/releng/jenkins-deploy!79dduvallmaster-I78b2c23e3d024a657463cc08c056b60126be2d08master
make-container-image: Generate new style image build reportrepos/releng/release!100dancymain-I3a091d41395419994bdcd8b31a864867bdb13506main
kubernetes.py: Use new image build report filerepos/releng/scap!396dancymaster-Ic14613d283d14b79d7e0e13cbf4ab1d5ef22b92fmaster
make-container-image: Add FORCE_MW_VERSION supportrepos/releng/release!95dancymain-Ic9799645fa0547b715efe48631ae788a4c9d436fmain
utils.py: read_wikiversions handle FORCE_MW_VERSIONrepos/releng/scap!382dancymaster-I8b5444f06b95856ba121d7f39025ede4f8248435master
make-container-image: Add MW_FORCE_VERSION supportrepos/releng/release!90dancymain-Ic9799645fa0547b715efe48631ae788a4c9d436fmain
Show related patches Customize query in GitLab

Event Timeline

bd808 triaged this task as Medium priority.
bd808 renamed this task from [WE6.2.1] Publish single version containers to [WE6.2.1] Publish pre-train single version containers.Jul 2 2024, 8:04 PM
bd808 updated the task description. (Show Details)
bd808 changed the subtype of this task from "Task" to "Feature Request".
bd808 set Due Date to Sep 30 2024, 11:59 PM.
bd808 raised the priority of this task from Medium to High.Jul 2 2024, 8:07 PM

Change #1053752 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/mediawiki-config@master] MWMultiVersion.php: Allow MW_FORCE_VERSION to pin the mw version

https://gerrit.wikimedia.org/r/1053752

Change #1053752 merged by jenkins-bot:

[operations/mediawiki-config@master] MWMultiVersion.php: Allow MW_FORCE_VERSION to pin the mw version

https://gerrit.wikimedia.org/r/1053752

Mentioned in SAL (#wikimedia-operations) [2024-07-22T14:59:30Z] <dancy@deploy1002> Started scap sync-world: Backport for [[gerrit:1053752|MWMultiVersion.php: Allow MW_FORCE_VERSION to pin the mw version (T369115)]]

Mentioned in SAL (#wikimedia-operations) [2024-07-22T15:01:50Z] <dancy@deploy1002> dancy: Backport for [[gerrit:1053752|MWMultiVersion.php: Allow MW_FORCE_VERSION to pin the mw version (T369115)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-07-22T15:08:41Z] <dancy@deploy1002> Finished scap: Backport for [[gerrit:1053752|MWMultiVersion.php: Allow MW_FORCE_VERSION to pin the mw version (T369115)]] (duration: 09m 10s)

Change #1056004 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/mediawiki-config@master] MWMultiVersion.php: Use FORCE_MW_VERSION instead of MW_FORCE_VERSION

https://gerrit.wikimedia.org/r/1056004

Change #1056004 merged by jenkins-bot:

[operations/mediawiki-config@master] MWMultiVersion.php: Use FORCE_MW_VERSION instead of MW_FORCE_VERSION

https://gerrit.wikimedia.org/r/1056004

Mentioned in SAL (#wikimedia-operations) [2024-07-22T19:34:02Z] <dancy@deploy1002> Started scap sync-world: Backport for [[gerrit:1056004|MWMultiVersion.php: Use FORCE_MW_VERSION instead of MW_FORCE_VERSION (T369115)]]

Mentioned in SAL (#wikimedia-operations) [2024-07-22T19:47:49Z] <dancy@deploy1002> dancy: Backport for [[gerrit:1056004|MWMultiVersion.php: Use FORCE_MW_VERSION instead of MW_FORCE_VERSION (T369115)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-07-22T19:54:24Z] <dancy@deploy1002> Finished scap: Backport for [[gerrit:1056004|MWMultiVersion.php: Use FORCE_MW_VERSION instead of MW_FORCE_VERSION (T369115)]] (duration: 20m 22s)

Change #1081281 had a related patch set uploaded (by Dduvall; author: Dduvall):

[operations/puppet@production] deployment_server::mediawiki: Execute scap mwscript/mwshell as mwbuilder

https://gerrit.wikimedia.org/r/1081281

dduvall merged https://gitlab.wikimedia.org/repos/releng/jenkins-deploy/-/merge_requests/84

jobs: Clean up l10n files using docker and copy all private settings

Change #1081281 merged by Scott French:

[operations/puppet@production] deployment_server::mediawiki: Execute scap mwscript/mwshell as mwbuilder

https://gerrit.wikimedia.org/r/1081281

bd808 assigned this task to dduvall.

See https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Group_-1/Progress_reports/2024-10-31 for the final report on this hypothesis. We are declaring the experiment a success and planning to follow up with work to put the new container to use in 2025.

CCiufo-WMF renamed this task from [WE6.2.1] Publish pre-train single version containers to [FY24-25 WE6.2.1] Publish pre-train single version containers.Sep 3 2025, 6:35 PM
bd808 changed the subtype of this task from "Feature Request" to "Goal".Sep 19 2025, 9:37 PM