Current situation

• Currently our versions such as wmde11 are just a suffix to the mediawiki version they are built on
• Users don't get any "meta info" based on the current numbering
• The current versioning scheme does NOT
  • Communicate breaking changes
  • Allow updating on a stable channel to get security fixes but prevent breaking changes
  • Provide an obvious way to communicate security fixes in dependencies while the MW version stays the same
  • Allow naming of cutting edge builds off of the upstream main branches (T343692)

Ideal situation

• We want to focus on how users see the versioning scheme
  • They need to be able to distinguish major versions from minor versions
  • They need to see whether a version includes breaking changes or not
  • They need to be able to follow patches while staying on a stable version without breaking changes
  • It should be easy to auto update following certain rules, such as "always run latest" or "stay on 3.x"
  • They need to know what to do when upgrading from one major version to another one
• Building using a certain version of the pipeline should create the same build artifacts (T340226) which get a unique versions assigned to them

Research

The built artifacts consist of components from loads of different sources with different versioning strategies:

Suggestion - Request For Comments

Versioning Scheme

We use SemVer (Semantic Versioning) https://semver.org/

• Breaking changes increment major version e.g. 2.3.1->3.0.0
• Feature releases increment minor version, e.g. 2.0.4->2.1.0
• Fixes, e.g. security fixes increment patch version, e.g. 2.0.0->2.0.1
• Pre-releases can be marked with an additional suffix following a hyphen (-), e.g. 3.0.0-alpha.20230411

We start with a major version of 2.0.0 (so this is bigger than current 1.39 mw version based releases).

What to version

We version the built artifacts, not the pipeline code. The pipeline code has a rolling release main branch. We use tags on the pipeline main branch to note which version of the pipeline builds which wikibase suite releases (see below).

All built artifacts of one version are together considered one wikibase suite release, that is:

• Container images (and tarballs - T343352) for MediaWiki+WikiBase+Extensions aka Bundle, WDQS, WDQS Frontend, etc.
• docker-compose files defining the whole stack ready to run
• env files configuring the whole stack ready to run (sane defaults, TODO: but what about passwords and keys?)
• see T340234 for more information

Upstream versions and usefulness of individual images

TODO Users of individual images that do not want to run a complete suite stack might be interested in the upstream version the build image is based on. Images built and published should be tagged with the upstream version number of the central software component. In case if the WDQS image, this would be the upstream WDQS version. In case of the WikibaseBundle image, this would probably be the mediawiki version the bundle is built on, we could also add some running number like a build date YYYYMMDD in order to communicate when the addon branches have been checked out to build this image.

TODO Should we tag the docker images with wikibase suite versions at all then?

Versions maintained in channels

There are three version channels maintained at every time: "stable", "oldstable" and "next".

• "stable", contains the current stable version, should be based on latest mediawiki stable most of the time
• "oldstable", contains the last stable version (was in stable channel before), still receives fixes
  • alternative approach: "lts", contains the current mediawiki lts release and all the other components maintained "as stable as possible" (mediawiki lts is maintained for 3 years https://www.mediawiki.org/wiki/Version_lifecycle) TODO: decision
• "next", contains the unstable version created off of the main branches of upstream components, contains mediawiki stable right after mediawiki release, until we are ready to release it as stable

The "next" builds allow us to be ready to release the next stable version soon after mediawiki releases the next stable version.

• Example at a certain point in time:
  • "stable" has 3.6.2, the current version, based on MW 1.40.4
  • "oldstable" has 2.8.1, the old version, based on MW 1.39.6
  • "next" has 4.0.0-pre.20230528, the upcoming unreleased version, based on MW main (TODO: we cannot use the date, can we? Can we build without updating the build code? Timer triggered github actions?)

Stable releases (in "stable" and "oldstable") do not include any breaking changes for the user within the same major version. We support them for a while (TODO: define how long). Breaking changes can happen at any time to the version currently in the "next" channel.

Bumping stable version

At a certain point in time we want to release a new major version. This is probably because major components brought in breaking changes, like a mediawiki minor version bump or maria db changing the on-disk file format. The upcoming wikibase suite version can be tested on the "next" channel. When we move close to the release, we should "freeze" the changes that land in "next" and only allow fixes. So we might refrain from pulling in upstream main changes during the freeze period. When we decide that the version in "next" channel is stable enough to become "stable" (TODO: how do we decide that?) the process is the following:

Before the version bump we have e.g.:

• "oldstable" has 2.8.1 the old/LTS version
• "stable" has 3.6.2 the current version
• "next" has 4.0.0-pre.20230528 the upcoming unreleased version

Bumping the version means:

• the version from "oldstable" does not receive any updates anymore, that is 2.8.1 is the last 2.X version in the example
• the version from "stable" becomes "oldstable", so the "oldstable" channel will from now on point to 3.6.2 and receive fixes there
• the version from "next" becomes "stable", so stable will point to the first version of 4.X, which is 4.0.0
• next resumes tracking upstream "main" branches and gets a new version number 5.0.0-pre.BUILDDATE

Tagging the pipeline main branch

In order to document, which version of the pipeline can build which wikibase suite versions, we use git tags on the main branch. Example: A single version f82a7cff of the main branch has the following three tags

• v2.8.1 the latest release of the old/LTS version (oldstable channel)
• v3.6.2 the latest release of the current version (stable channel)
• v4.0.0-pre.20230528 the latest build of the upcoming unreleased version (next channel) (TODO: we cannot use the date, can we? Can we build without updating the build code?)

This is because this single pipeline version can build all latest versions from all supported channels ("stable", "oldstable", "next") at the same time.

Distribution

The wikibase suite distribution consists of two kinds of things. A set of files and a set of docker images. The files will be distributed as a git repo. The docker images will be distributed via a docker registry.

All docker images are tagged with the version and the channel name of the version they are created for. Lets take the example from above again. Each of the docker images belonging to the release will be tagged as follows:

For "oldstable" 2.8.1 the old/LTS version, all images will have the following tags:

• "2.8.1" to have a tag for the full version number, the indivitual image identifying tag that never changes
• "2.8" to have a tag that will be reused for patch versions of the same minor version
• "2" to have a tag that will be reused for minor versions of the same major version
• "oldstable" to have a tag that follows the channel and is reused as soon as the channel updates

For "stable" 3.0.0 the current version, all images will have: "3.6.2", "3.6", "3" and "stable". It is the same pattern as for "oldstable".

For "next" 4.0.0-pre.20230528 the upcoming unreleased version, all images will have the following tags:

• "4.0.0-pre.20230528" to have a tag for the full version number, the individual image identifying tag that never changes
• "4" to have a tag that will be reused for this major version
• "next" to have a tag that follows the channel and is reused as soon as the channel updates

TODO: Should we really tag the channel names? What is the use case for that?

TODO: how do we version the files relating to the release? Probably in the same scheme, just using git tags and branches T340234

Updating when only one tiny bit changed

When we version all artifacts together, a small change in one image will lead to all containers getting a version bump, even though their actual content did not change. As long as our builds are not perfectly reproducible, we will generate new images with the same content, tag it with the new version and trigger a redownload for our users. This is a waste of resources. Using individual versions for the actual container images would fix that problem and we would only update what changed.

Product Question

@jon_amar-WMDE

• Do we release one bundle that is meant to work as a whole?

OR

• Do we release a set of loose components that are meant to work on their own?

TODO go through, wrap head around it https://www.wikidata.org/wiki/Wikidata:Stable_Interface_Policy

How could we leverage git branches?

https://app.conceptboard.com/board/u723-u2hs-23mc-sbxk-i7zn