ccing @thcipriani hoping he could direct our questions to someone from his team that might have a few minutes to read those. thanks!

Is it possible for Wikibase release tarballs (i.e. Wikibase extension, other MW extensions, and non-extension software) to be stored on releases.wikimedia.org ?

I'd presume there's no reason why not.. Any reason to not use ExtensionDistributor for Wikibase/MW extensions? Does ExtensionDistributor work with tags? Or are you planning to create bundles etc?

I'm not saying this is the right way (point of reference for a similar example), but the MediaWiki Language Extension Bundle ones seemingly end up on translatewiki.net - https://www.mediawiki.org/wiki/MediaWiki_Language_Extension_Bundle maybe this should be addressed at some later time

How would we get access to publishing these artifacts?

SRE-Access-Requests for releasers-mediawiki (I don't suspect we need a new group). Easier done for anyone already with deployment access

https://github.com/wikimedia/puppet/blob/fa1cb27467534f8bedf6d029d9c1e868149a2412/modules/admin/data/data.yaml#L125-L131

Thank you for the reply!

Any reason to not use ExtensionDistributor for Wikibase/MW extensions? ... Or are you planning to create bundles etc?

The pipeline we are working on is indeed containing some extensions but some are not (WDQS, WDQS-UI etc), you could see it as a bundle of sorts but they would still be presented as separate components that we with the highest confidence possible can guarantee work together.

These components would have gone through an additional acceptance-testing step in order to guarantee compatibility between the them. The end result of this would be tarballs, the release notes we discussed earlier today and docker images with some of these components bundled together. If we would have the option of hosting the tarballs on https://extdist.wmflabs.org/dist/extensions/ (effectively overriding the tarball it is currently pointing to) I guess this would be an option as well, however I think the end-goal is to regain a bit of control over the release process and the presentation of the Wikibase "Suite" of software that is required / compatible to work together.

Does ExtensionDistributor work with tags?

From what I can tell it currently does not.

In T268345#6646520, @toan wrote:

Does ExtensionDistributor work with tags?

From what I can tell it currently does not.

T70939: ExtensionDistributor: Support custom tags for extensions

Of course there was a bug for it :)

Could be worth WMDE putting a bit of time (I imagine it's not going to need too much engineering effort, probably, maybe...) into improving ExtensionDistributor to add that tag support

Just putting some ideas out there. I'm definitely not saying "YOU MUST DO IT THIS WAY" :). But at the same time, there's maybe some value for having the tags in the repos (which presumably will exist as part of the building process), and a way to get them from extension disitributor too.

Certainly, if it's more "complete" releases/collections, I don't see any obvious reason it can't go on releases.wikimedia.org

I guess another question is how you want them to go there. Manually? Or as the output of some sort of CI process, and therefore "automatic"?

Is it possible for Wikibase release tarballs (i.e. Wikibase extension, other MW extensions, and non-extension software) to be stored on releases.wikimedia.org ?

I have no objections.

How would we get access to publishing these artifacts?

What Reedy said about releasers-mediawiki would work.

You could also add a group just for this following this example to create a group specifically for this: https://gerrit.wikimedia.org/r/c/operations/puppet/+/483800

Could be worth WMDE putting a bit of time (I imagine it's not going to need too much engineering effort, probably, maybe...) into improving ExtensionDistributor to add that tag support

Just putting some ideas out there. I'm definitely not saying "YOU MUST DO IT THIS WAY" :). But at the same time, there's maybe some value for having the tags in the repos (which >presumably will exist as part of the building process), and a way to get them from extension disitributor too.

I think this would be desirable as well but I would look to my EM / PM:s (@WMDE-leszek / @Samantha_Alipio_WMDE ) to decide if this is something we would work on at this stage. Having a unified way of getting extensions is a great idea but right now we are looking for a place host not only extensions.

I guess another question is how you want them to go there. Manually? Or as the output of some sort of CI process, and therefore "automatic"?

This is a bit unclear to me and is under investigation (https://phabricator.wikimedia.org/T267893), currently the MVP workflow is run on github actions but there is a desire to move to using PipelineLib or some other infrastructure owned by the foundation or WMDE (https://phabricator.wikimedia.org/T268402). I think we would like to automate as much as possible but there is a chance the actual publishing (file transfer) would need to be manual.

Another topic that has come up multiple times during our discussions is why the vendor folder is included in these extension tarballs delivered by ExtensionDistributor, shouldn't this be installed by composer and merged by the composer-merge-plugin? There seems to be a lot of different arguments for including vendor but also some for leaving it out. Does anyone know the reasoning behind that?

In T268345#6648180, @toan wrote:

Another topic that has come up multiple times during our discussions is why the vendor folder is included in these extension tarballs delivered by ExtensionDistributor, shouldn't this be installed by composer and merged by the composer-merge-plugin? There seems to be a lot of different arguments for including vendor but also some for leaving it out. Does anyone know the reasoning behind that?

Not everyone can run composer (security, shared hosting, shell access etc), and so we support both vendor in the extension folder (for an extensions dependancies), as well as composer-merge plugin method.

In some cases, where they have many extensions installed on a wiki farm, and don't want a huge vendor folder, this helps here as they include them only when they are loading that extension.

So in a similar way to the MW tarballs with vendor included, we do for extensions.

It's easier to rm -rf vendor and the composer.lock (if you don't need them) than it is to add them later

Task for this is/was T70940: ExtensionDistributor: package composer dependencies inside tarballs

Not everyone can run composer (security, shared hosting, shell access etc), and so we support both vendor in the extension folder (for an extensions dependancies), as well as composer-merge plugin method.

Thank you for this explanation, sounds very reasonable.

You could also add a group just for this following this example to create a group specifically for this: https://gerrit.wikimedia.org/r/c/operations/puppet/+/483800

And thank you for @thcipriani for the pointers, much appreciated.

This open task is only tagged with the RelEng Q2 project tag which recently has been archived. Please add an active project tag to this task so this task is discoverable. Thanks!