https://github.com/wmde/wikibase-release-pipeline/pull/901

Hi Adam,

Thanks for this suggestion, we think this is very helpful. We are currently working on getting this out in the next release.

Regards,
Deepesha

Sorry my ignorance but, what's the difference between Wikibase Suite and Wikibase Deploy?

Proposed Version Reporting Solution

I think we should extend Adam’s work in #901 to capture version/build provenance for diagnostics and analytics. So we keep the Special:Version > "Installed software" approach from PR #901 which I think is a good call and a solid foundation, but we gather and set the following versions automatically at build time:

• Wikibase Suite Docker Image (https://www.mediawiki.org/wiki/Wikibase/Suite): <wikibase image version>
• Wikibase Suite Build Tools (https://github.com/wmde/wikibase-release-pipeline): <pipeline git sha> (build tools revision used to build the image)

And set the following manually when as part of release we update the deploy/package.json version:

• Wikibase Suite Deploy (https://www.mediawiki.org/wiki/Wikibase/Suite/Deploy): <deploy version> (only when Deploy is used)

Also I propose we expose a small, flat JSON document for scraping as a backup to avoid HTML parsing and as a backup means for accessing the running versions:

• /<wiki>/wbs-versions.json (e.g. https://example.org/wbs-versions.json)

Example response shape:

json
{
  "wikibase_image_version": "5.0.0",
  "deploy_version": "5.0.1",
  "pipeline_git_sha": "cc32494ebf8a"
}

Estimated Engineering Effort:

1-2 days + code review time.

Notes:

• I don't think we should attempt to report versions of the adjacent services (WDQS, QuickStatements, Elasticsearch, MariaDB) for now. While we could add custom code for version reporting to each of those images, their endpoints are not predictable or necessarily published, and passing their versions into the Wikibase container via Docker Compose would be unreliable if users customize the compose setup (values can go missing or become incorrect). By contrast, the Wikibase image version is intrinsic to the container, and the Deploy version is a single, explicitly-set signal when running via the Deploy stack. If we assume an unmodified and standard Deploy setup these other versions can be inferred from the Build Tools (pipeline) commit hash which would be reported.
• The existing “pingback”/callback registers new instances with the metadata service on first install. Once wbs-versions.json exists, the metadata service can fetch it from the base URL it already knows. Alternatively, we can as of this change include these values directly in the callback to avoid the extra request but would require a change to the GraphQL endpoint to allow for those extra fields. I think we should hold off on this for now, that said if it is compelling to add this it would not be consequential extra effort.
• Building on what’s here, it may be worth adding a dedicated “Wikibase Suite” special page later—implemented as a full extension bundled with the Wikibase image—that users can visit any time after installation to announce their Wikibase (e.g., via an “Announce” / Wikibase Metadata (Pingback) button). The page could also include a brief explanation of what announcing does and links to other relevant Wikibase Suite documentation. This could be added in addition to, or instead of, a Wikibase Suite install landing page (whether users install via the Docker image alone or via a Deploy package).

Hey @Olea!

• Wikibase Suite is the name of our team, and also our "product line", so it is like a prefix to everything (e.g. Wikibase Suite Wikibase Docker Image, Wikibase Suite Deploy)
• Wikibase Suite Deploy is the name of the specific product that lives at https://github.com/wmde/wikibase-release-pipeline/tree/main/deploy and is the full Docker Compose based configuration for running all or the Docker images we publish together as a full "suite" of running services.

Does that make sense? Also, please anyone else on the team correct me if I've gotten any of that wrong.

@lojo_wmde somewhat of a drive-by thought but I wonder if https://www.mediawiki.org/wiki/Special:ApiSandbox#action=query&format=json&meta=siteinfo&formatversion=2&siprop=extensions%7Clibraries%7Cgeneral has some of what you need in a json parseable format to save you scraping Special:Version or having to roll another source of metadata about the site

Hi @Tarrow! Drive-bys welcome :) Here is my full answer to that: Currently if we followed Adam's lead and didn't create a proper extension but just hooks that insert those Installed software rows, we wouldn't get anything back from ActionAPI for this. However, I am leaning towards implementing as an extension as I think it is a good platform for additional "Wikibase Suite" specific features running natively in our Wikibase Image, and isn't a lot more code. If I do that then likely I can perhaps inject / sneak all three of the versions into that extension version and therefore get to it from ActionAPI... That all said the proposal as it stands also generates a static /wbs-versions.json file at image build time which makes the scraping also unnecessary. My additional note was just to say we could also send those versions as part of the callback if we updated our Wikibase Metadata endpoint to accept them and then there would be no round trip required, but either way not scraping. Basically as spec'd even without a full extension we have a good non-scraping option in the json file.

@Leif_WMDE note that I am shifting as I think about this more to making this a full extension vs just the hooks plan we get from Adam's work. If I get the go ahead on that then I'd like to give you a sense of the capabilities / implications of that are. Basically all positives and after having looked into that option more, I don't see it as any consequential additional scope.

Updated PRD/Proposal with what we're moving ahead on for now https://hackmd.io/@lorenjohnson/BJvl6DmvZe

Currently if we followed Adam's lead and didn't create a proper extension but just hooks that insert those Installed software rows, we wouldn't get anything back from ActionAPI for this.

I expect that the software rows and can probably should be added to the API output of MediaWiki if they are not already there

I don't think we should attempt to report versions of the adjacent services (WDQS, QuickStatements, Elasticsearch, MariaDB) for now.

See T108226: Special:Version should show the versions of services that a wiki uses as a cross-link that could be relevant, and would also tie into the API.

Maybe there is or should be some overlap with https://www.mediawiki.org/wiki/Extension:WikibaseManifest rather than creating yet another wikibase extension, though this is starting to go down a rabbit hole.

As the original author, there is certainly scope creep from the initial request, which is a MVP of being able to see at all if a wikibase comes from 1) a "wikibase suite wikibase docker image" and 2) is running using "wikibase suite wikibase deploy"

Having it on a special page works for that in an initial requirement from me and can likely be exposed via API in an existing format with a small MediaWiki change.

Trying to write the original description up as a story

As an observer of growth of the wikibase ecosystem,
I want to be able to determine the installation method of a public wikibase via publicly available means

I believe the main options within, or matrix of possibilities are

• container image / not container image (do it really matter, probably not)
• cloud host, or own host (again pretty easy to figure out most of the time anyway)
• git or tarball (probably doesn't really matter, but can already be inferred via the git info on Special:Version (or lack thereof)
• wikibase suite / wikibase dpeloy or not (The hardest thing to determine currently)

So rewriting it for the hardest thing to determine currently

As an observer of growth of the wikibase ecosystem,
I want to be able to determine if a wikibase installation uses a build from the "wikibase suite" product (wikibase-release-pipeline)

and

As an observer of growth of the wikibase ecosystem,
I want to be able to determine if a wikibase installation uses the "wikibase suite wikibase deploy" to deploy itself

/wbs-versions.json would be sub optimal for me personally as a solution, as its yet another request to periodically make for every wikibase while scraping.
Ideally such information could be surfaced somewhere that similar information is already presented.

Which I guess would lead me to https://www.mediawiki.org/wiki/Manual:Hooks/APIQuerySiteInfoGeneralInfo rather than some static file somewhere, even if it contains the same content.
Which would make it appear here https://www.mediawiki.org/wiki/Special:ApiSandbox#action=query&format=json&meta=siteinfo&formatversion=2&siprop=general