Seeking guidance from Release-Engineering-Team on the best approach for this. Thanks!

So, to unblock this slightly and a few other things (like preparing phan patches, even if they're sent to github, and unblocking CI for Hallo Welt working on the skin/extension)...

I think I'm right in saying we will not be deploying the extensions (Chameleon and Bootstrap) directly from github. That's a non starter.

So that means we need local mirrors whatever else happens (how syncing/maintaining here is done is TBD as per this ticket). Chameleon was previously in Gerrit, but went away. I'm hoping that means the archived repo still has common ancestory.

I'm going to unarchive the Chameleon repo, push the latest version from GitHub there, ideally remove C+2 rights (to prevent accidental merging and conflicts), and re-enable CI on the repo.

Similar for Bootstrap, which seems to be here and archived too.

I don't see any reason we need to do the same for the vendor-included stuff (as we don't for anything else, unless we've forked it from lack of maintainership etc). So mediawiki/scss and scssphp/scssphp will be left alone... And CI should just include them as it goes through and things should just work... In theory :)

Bootstrap and Chameleon repos re-synced to gerrit

CI patches in https://gerrit.wikimedia.org/r/#/c/integration/config/+/597810/

Thank you! That's great progress!

The question remains how to keep the two repos synced. What is the best workflow for that?

In T252463#6155814, @CCicalese_WMF wrote:

Thank you! That's great progress!

The question remains how to keep the two repos synced. What is the best workflow for that?

Really not sure.

I guess we should also decide if we want to follow master of those repos, or specific release tags?

The latter would require some changes to the make-release script, but also makes them then a special case.

Just noticed a bit of a gotcha. The existing Chameleon repo is lowercased, so mediawiki/skins/chameleon not mediawiki/skins/Chameleon which causes problems like https://gerrit.wikimedia.org/r/#/c/mediawiki/skins/WikimediaApiPortal/+/588671/7/skin.json here

Is it worth creating a captialised version, archive the lowercased version before we continue much further?

It looks like the github repo also is lowercase. See https://github.com/ProfessionalWiki/chameleon/issues/149, in which the fix went to lowercase. Would having a different casing have other implications, including issues with keeping the two repos synced? @JeroenDeDauw, thoughts?

Oh, is it?

It's hard to tell with GitHub as it is case-insensitive https://github.com/ProfessionalWiki/Chameleon and https://github.com/ProfessionalWiki/chameleon - both give a HTTP 200

IMHO, then https://github.com/ProfessionalWiki/chameleon/blob/85a3492/skin.json#L2 should possibly be lowercase (I'm sure with other extensions we've had issues with this, but might've been spaces vs no spaces, rather than capitalisation

For manual syncing it definitely doesn't matter, you can add whatever git remotes we want, whatever the name. If we do something more automatic, I think it shouldn't matter either, as you usually explicitly define the remote; so it can be https://github.com/somename/aardvark for all git cares etc

Aha, I see the emphasis and mentions in https://github.com/wikimedia/mediawiki-skins-Chameleon/blob/master/docs/installation.md too

In T252463#6156045, @Reedy wrote:

IMHO, then https://github.com/ProfessionalWiki/chameleon/blob/85a3492/skin.json#L2 should possibly be lowercase (I'm sure with other extensions we've had issues with this, but might've been spaces vs no spaces, rather than capitalisation

Ok, so it's not just IMHO, it's definitely needed otherwise things break

For CI dependancy, if the case is wrong

17:51:24 A dependency error was encountered while installing the extension "WikimediaApiPortal": Could not find the registration file for the extension "Chameleon

On the wiki if the case doesn't match

( ! ) Fatal error: Uncaught ExtensionDependencyError: WikimediaApiPortal requires chameleon to be installed. in /var/www/wiki/mediawiki/core/includes/registration/ExtensionRegistry.php on line 398

In T252463#6155869, @Reedy wrote:

In T252463#6155814, @CCicalese_WMF wrote:

Thank you! That's great progress!

The question remains how to keep the two repos synced. What is the best workflow for that?

Really not sure.

I guess we should also decide if we want to follow master of those repos, or specific release tags?

The latter would require some changes to the make-release script, but also makes them then a special case.

There are a few options that I'm aware of:

1. Use gerrit for development and gerrit can mirror to github automatically
2. Development stays on github and every so often a human/group of humans (with the forge committer permisison) reviews changes and manually pushes them to a gerrit upstream
3. Development stays on github and code is automatically mirrored to gerrit.

From a maintenance perspective (from the releng side anyway) option 1 is the best -- no special permissions, no mirroring to monitor.

There may be other considerations here that make option 2 a better one; i.e., it may be that a person needs to manually ensure compatibility between Chameleon's composer dependencies and the dependencies in mediawiki/vendor, for example (something @Jdforrester-WMF mentioned in IRC as a consideration).

In T252463#6156277, @Reedy wrote:

In T252463#6156045, @Reedy wrote:

IMHO, then https://github.com/ProfessionalWiki/chameleon/blob/85a3492/skin.json#L2 should possibly be lowercase (I'm sure with other extensions we've had issues with this, but might've been spaces vs no spaces, rather than capitalisation

Ok, so it's not just IMHO, it's definitely needed otherwise things break

For CI dependancy, if the case is wrong

17:51:24 A dependency error was encountered while installing the extension "WikimediaApiPortal": Could not find the registration file for the extension "Chameleon

On the wiki if the case doesn't match

( ! ) Fatal error: Uncaught ExtensionDependencyError: WikimediaApiPortal requires chameleon to be installed. in /var/www/wiki/mediawiki/core/includes/registration/ExtensionRegistry.php on line 398

I've upstreamed https://github.com/ProfessionalWiki/chameleon/pull/155 to make the canonical name lowercase to match the dir, but also set namemsg to use an i18n string which is capitalised for display purposes

I've got a hunch it might be related to the issues in T253331: WikimediaApiPortal results in CI OOM-ing

In T252463#6156889, @thcipriani wrote:

There are a few options that I'm aware of:

1. Use gerrit for development and gerrit can mirror to github automatically
2. Development stays on github and every so often a human/group of humans (with the forge committer permisison) reviews changes and manually pushes them to a gerrit upstream
3. Development stays on github and code is automatically mirrored to gerrit.

From a maintenance perspective (from the releng side anyway) option 1 is the best -- no special permissions, no mirroring to monitor.

There may be other considerations here that make option 2 a better one; i.e., it may be that a person needs to manually ensure compatibility between Chameleon's composer dependencies and the dependencies in mediawiki/vendor, for example (something @Jdforrester-WMF mentioned in IRC as a consideration).

For no 1, that'd need to be on the maintainers of the extension to decide to do that. They moved away before, so I imagine they might not want to come back. And ideally we don't want to end up with a fork.

For 2 and 3, as I think I mentioned earlier in this task, it probably depends whether we want to follow master or stable releases. If the latter, we can just do manual pushes as necessary, and then bump the version we're using etc, no major issue there.

For the libraries, the skin doesn't depend on anything we already include in vendor, so should be no conflicts there, but obviously it can bump it's own dependancies, and those in the sub-sub (Chameleon skin depends on Bootstrap extension, depends on mediawiki/scss library, depends on scssphp/scssphp) - though, based on T252462#6131830, Bootstrap and the scss dependancies potentially aren't needed anyway. Which simplifies stuff

We're already half-forking the stack with WikimediaApiPortal; we could just fold the code into that and simplify everything massively?

In T252463#6159044, @Jdforrester-WMF wrote:

We're already half-forking the stack with WikimediaApiPortal; we could just fold the code into that and simplify everything massively?

Technically I'm sure.

Contract/SOW wise? I've no idea. I don't know what's in there, in scope etc

In T252463#6159044, @Jdforrester-WMF wrote:

We're already half-forking the stack with WikimediaApiPortal; we could just fold the code into that and simplify everything massively?

WikimediaApiPortal extends the generic Chameleon skin to incrementally add UI components that are specific to the API Portal and Wikimedia environment. Chameleon handles the bulk of the skin functionality, while WikimediaApiPortal adds elements such as the Wikimedia footer and specifies styling conforming with the Wikimedia Design Style Guide. It would be counterproductive to pull all of Chameleon and Bootstrap into WikimediaApiPortal, increasing the maintenance burden, when Chameleon is a widely-deployed and actively-maintained skin.

In T252463#6156889, @thcipriani wrote:

There are a few options that I'm aware of:

1. Use gerrit for development and gerrit can mirror to github automatically
2. Development stays on github and every so often a human/group of humans (with the forge committer permisison) reviews changes and manually pushes them to a gerrit upstream
3. Development stays on github and code is automatically mirrored to gerrit.

From a maintenance perspective (from the releng side anyway) option 1 is the best -- no special permissions, no mirroring to monitor.

There may be other considerations here that make option 2 a better one; i.e., it may be that a person needs to manually ensure compatibility between Chameleon's composer dependencies and the dependencies in mediawiki/vendor, for example (something @Jdforrester-WMF mentioned in IRC as a consideration).

I've discussed option 1 with the Chameleon maintainer, and it is a non-starter. Either 2 or 3 seem reasonable. Note that I was recently added as a contributor to the Chameleon and Bootstrap github repos.

In T252463#6159156, @CCicalese_WMF wrote:

In T252463#6156889, @thcipriani wrote:

There are a few options that I'm aware of:

1. Use gerrit for development and gerrit can mirror to github automatically
2. Development stays on github and every so often a human/group of humans (with the forge committer permisison) reviews changes and manually pushes them to a gerrit upstream
3. Development stays on github and code is automatically mirrored to gerrit.

From a maintenance perspective (from the releng side anyway) option 1 is the best -- no special permissions, no mirroring to monitor.

There may be other considerations here that make option 2 a better one; i.e., it may be that a person needs to manually ensure compatibility between Chameleon's composer dependencies and the dependencies in mediawiki/vendor, for example (something @Jdforrester-WMF mentioned in IRC as a consideration).

I've discussed option 1 with the Chameleon maintainer, and it is a non-starter. Either 2 or 3 seem reasonable. Note that I was recently added as a contributor to the Chameleon and Bootstrap github repos.

Creating a gerrit group of folks who can push manually mirror stable release is totally doable if that's the path we want to go down. Following master automatically is a bit of a rube-goldberg system -- can be flaky -- but also doable.

I'd recommend option 2 as well. (Although by pushing a merge commit for review, not force-pushing directly).

Option 3 would mean bypass our CI which would then require various other stuff to be set up. It also makes for a natural place for code review and other ad-hoc communication with upstream/vendor constractors to take place prior to deployment etc.

In T252463#6159823, @Krinkle wrote:

Option 3 would mean bypass our CI which would then require various other stuff to be set up.

So would option 2 if we were doing direct pushes... Which we kinda need to do unless we're going to create diverging commits that add a change-id... Which adds complexity, makes it slightly a fork as the commits will have different hashes to upstream etc

I know there's a Require Change-Id in commit message option in Gerrit... I don't know how well turning it off works; does it work? Have we tried etc?

In T252463#6160120, @Reedy wrote:

In T252463#6159823, @Krinkle wrote:

Option 3 would mean bypass our CI which would then require various other stuff to be set up.

So would option 2 if we were doing direct pushes... Which we kinda need to do unless we're going to create diverging commits that add a change-id... Which adds complexity, makes it slightly a fork as the commits will have different hashes to upstream etc

I know there's a Require Change-Id in commit message option in Gerrit... I don't know how well turning it off works; does it work? Have we tried etc?

Afaik it's fine to only require it for the merge commits, the "real" commits would be inherited as-is without any rebase or rehashing.

So, it sounds like we're converging on:

Development stays on github and every so often a human/group of humans (with the forge committer permisison) reviews changes and manually pushes them to a gerrit upstream

The process should be documented and, if possible, scripted as much as possible to avoid inconsistencies and to ensure that it isn't an onerous process that discourages keeping the gerrit repo up to date.

Given that the github commits would retain their hashes, would there be any gotchas in comparing the two repos to ensure that there is no divergence?

It would be good to enforce a workflow where patches are submitted on github and make their way through the process to gerrit rather than patching directly on gerrit unless there is urgent need for a hotfix. But, there would be implications for libraryupgrader and localization.

Are we planning on following master? Or a specific branch/release?

I don't think there's any problem doing hotfixes onto the wmf deployment branches that will exist in the gerrit "mirror" of the repo, and then syncing upstream to master where needed. That shouldn't cause any problems

I also don't think we need to worry about l10n/i18n much. There aren't many messages in the repo, and it looks like translatewiki only manages to push less than one commit a month on average - https://github.com/ProfessionalWiki/chameleon/commits/master/resources/i18n. Which shows new messages aren't really added or the existing ones changed much; so they're fairly stable. Sure, when it hits Wikimedia it might get a bit more translation as people start filling in the gaps, but there's not many messages, so I don't think it'd take most translators very long to get them done.

Library upgrader is a bit more of an issue, but syncing those over to github for merging shouldn't be a big issue. Even if they're created as PR on gerrit then moved over.

In T252463#6163471, @Krinkle wrote:

In T252463#6160120, @Reedy wrote:

In T252463#6159823, @Krinkle wrote:

Option 3 would mean bypass our CI which would then require various other stuff to be set up.

Are there CI needs here?

So would option 2 if we were doing direct pushes... Which we kinda need to do unless we're going to create diverging commits that add a change-id... Which adds complexity, makes it slightly a fork as the commits will have different hashes to upstream etc

I know there's a Require Change-Id in commit message option in Gerrit... I don't know how well turning it off works; does it work? Have we tried etc?

Afaik it's fine to only require it for the merge commits, the "real" commits would be inherited as-is without any rebase or rehashing.

^ this is the case. Change-id required for things going through review, but not direct to the repo. I have never tried turning it off. It's a variable on the project config page ("Require Change-Id in commit message:") https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/skins/chameleon

LibUp would also need to be disabled: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Bootstrap/+/601177/

In T252463#6181032, @DannyS712 wrote:

LibUp would also need to be disabled: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Bootstrap/+/601177/

Maybe, maybe not. While that change itself is useless (adding vendor)... They can still be upstreamed