Page MenuHomePhabricator

Migrate metrics-platform repo to GitLab
Closed, ResolvedPublic8 Estimated Story Points

Description

Event Timeline

thcipriani triaged this task as Medium priority.Sep 5 2023, 3:13 PM

Looks like we're running a series of test entrypoints (npm, composer, maven, and swift). Release-Engineering-Team can help with translating these from Gerrit/Zuul ā†’ GitLab. We'd need some support and consulting time to ensure they're working as they were before.

VirginiaPoundstone lowered the priority of this task from Medium to Low.Sep 12 2023, 1:47 PM

Change 958901 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/extensions/EventLogging@master] modules/lib: Use GitLab URL for lib/metrics-platform

https://gerrit.wikimedia.org/r/958901

The wikimedia/metrics-platform Packagist project is now being published to from GitLab. Thanks, @Reedy!

@thcipriani: Is there anything to be done with the mirror on GitHub?

@thcipriani: Is there anything to be done with the mirror on GitHub?

It's an interesting question more widely too...

Do we want to keep them? Have we got ways to make the replication work currently (ideally gitlab -> github, without going via gerrit)?

I've reconfigured the wikimedia/metrics-platform package at https://packagist.org/packages/wikimedia/metrics-platform back from GitLab to github.com because it seems one of GitLab and Composer doesn'tt support the other to produce a dist package, and thus rebuilding mediawiki-bendor became a mess even for older versions.

https://gerrit.wikimedia.org/r/c/mediawiki/vendor/+/960127/1

Afaik we're not supposed to use GitLab for production MediaWiki platform repositories yet, which libraries are a part of. In any event, this one didn't work. Maybe there's a way it can work. I've reverted it in order to unbreak mediawiki-vendor.

@thcipriani: Is there anything to be done with the mirror on GitHub?

It's an interesting question more widely too...

Do we want to keep them? Have we got ways to make the replication work currently (ideally gitlab -> github, without going via gerrit)?

We do want to keep them. For visibility and internet points purposes.

There are ways to make it work in GitLab (owners can mirror under https://gitlab.wikimedia.org/repos/data-engineering/metrics-platform/-/settings/repository ). But, as part of archiving, we have to change the phabricator remote to stop mirroring gerrit, having that as a central place to mirror seems like an easier path.


I've reconfigured the wikimedia/metrics-platform package at https://packagist.org/packages/wikimedia/metrics-platform back from GitLab to github.com because it seems one of GitLab and Composer doesn'tt support the other to produce a dist package, and thus rebuilding mediawiki-bendor became a mess even for older versions.

https://gerrit.wikimedia.org/r/c/mediawiki/vendor/+/960127/1

Afaik we're not supposed to use GitLab for production MediaWiki platform repositories yet, which libraries are a part of. In any event, this one didn't work. Maybe there's a way it can work. I've reverted it in order to unbreak mediawiki-vendor.

Odd. GitLab can produce a zip (in this case: https://gitlab.wikimedia.org/api/v4/projects/1618/repository/archive.zip?sha=main )

This might have something to do with the gitlab-domains variable. Looking at the source code it seems to know how to handle github.com and gitlab.com based on the host url, but with self-hosted gitlab installs it seems to need that config (see updateDistReference in composer source)

But, as part of archiving, we have to change the phabricator remote to stop mirroring gerrit, having that as a central place to mirror seems like an easier path.

This should be done now. Phabricator is observing GitLab and mirroring to GitHub (or, at least, that's how it's now configuredā€”I'll keep an eye on it to see if it works).

I've reconfigured the wikimedia/metrics-platform package at https://packagist.org/packages/wikimedia/metrics-platform back from GitLab to github.com because it seems one of GitLab and Composer doesn'tt support the other to produce a dist package, and thus rebuilding mediawiki-bendor became a mess even for older versions.

Thanks.

Afaik we're not supposed to use GitLab for production MediaWiki platform repositories yetā€¦

Is this policy documented somewhere I can reference? Also, does it cover development? If so, then we'll revert back to using Gerrit quickly and easily.

After creating https://gerrit.wikimedia.org/r/c/mediawiki/libs/metrics-platform/+/964595... It's unclear where patches should currently go, and that there is some divergence between https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/libs/metrics-platform/+log and https://gitlab.wikimedia.org/repos/data-engineering/metrics-platform/-/commits/main/

Master on gerrit has https://gerrit.wikimedia.org/r/c/mediawiki/libs/metrics-platform/+/962080 which isn't in main...

Gitlab has a few commits that aren't on gerrit master...

And there's nothing in the gerrit description that the repo shouldn't be used, nor is it readonly or similar (which may be hard if we're replicating to the repo)...

Plus CI still works over there...

Plus CI still works over there...

CI should work on GitLab (main and branches from main) and on Gerrit (master and branches from master). I think it's not working for your commit because the .gitattributes file isn't covered by any of the CI jobs (see https://gerrit.wikimedia.org/g/integration/config/+/50e046a09c9287a266183ad791afd9dad4a8027a/zuul/layout.yaml#1294) šŸ¤”


There are a couple of MRs open on GitLab. I'm happy to converge the two branches and shift us back to Gerrit once they're merged.

I've reconfigured the wikimedia/metrics-platform package at https://packagist.org/packages/wikimedia/metrics-platform back from GitLab to github.com because it seems one of GitLab and Composer doesn'tt support the other to produce a dist package, and thus rebuilding mediawiki-bendor became a mess even for older versions.

Thanks.

Afaik we're not supposed to use GitLab for production MediaWiki platform repositories yetā€¦

Is this policy documented somewhere I can reference? Also, does it cover development? If so, then we'll revert back to using Gerrit quickly and easily.

For work that needs to be coordinated via Depends Onā€”extensions, skins, coreā€”we need to keep those on Gerrit at the moment. If the repo is not blocked by that, then it can migrate to GitLab.


After creating https://gerrit.wikimedia.org/r/c/mediawiki/libs/metrics-platform/+/964595... It's unclear where patches should currently go, and that there is some divergence between https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/libs/metrics-platform/+log and https://gitlab.wikimedia.org/repos/data-engineering/metrics-platform/-/commits/main/

Master on gerrit has https://gerrit.wikimedia.org/r/c/mediawiki/libs/metrics-platform/+/962080 which isn't in main...

Gitlab has a few commits that aren't on gerrit master...

And there's nothing in the gerrit description that the repo shouldn't be used, nor is it readonly or similar (which may be hard if we're replicating to the repo)...

Plus CI still works over there...

I have refrained from archiving this repository on Gerrit until issues were settled here.


So there are two paths:

If this repo benefits heavily from being referenced by Depends On then we can shift back to Gerrit.
If not. Then I can archive Gerrit, close out patches, remove from CI, and update the GitHub repo's mainline branch to main (already updated the diffusion mirroring).

So there are two paths:

If this repo benefits heavily from being referenced by Depends On then we can shift back to Gerrit.
If not. Then I can archive Gerrit, close out patches, remove from CI, and update the GitHub repo's mainline branch to main (already updated the diffusion mirroring).

I'd presume that it wouldn't directly.

If there are changes required in the EventLogging extension, due to changes in this library... This will require bumping this library in vendor, would depend-on a vendor patch bumping it, not specifically on a/the underlying change in this repo itself.

So, it sounds like it'd be fine to:

  • Archive the gerrit repo with a pointer to GitLab
  • Close patches on gerrit with a pointer to the new GitLab repo
  • Remove the project from zuul ci
  • Repoint the GitHub repo from master ā†’ main

Does that sound like the right path @phuedx ? Would I be stepping on your toes?

Does that sound like the right path @phuedx ? Would I be stepping on your toes?

We should also mirror the project to GitHub so that we don't break updating the mediawiki-vendor repository (per T344733#9192434). With that done, I think that those steps are correct.

Change 965794 had a related patch set uploaded (by Thcipriani; author: Thcipriani):

[integration/config@master] Zuul: archive mediawiki/libs/metrics-platform

https://gerrit.wikimedia.org/r/965794

Change 965796 had a related patch set uploaded (by Thcipriani; author: Thcipriani):

[integration/config@master] Remove metrics-platform jobs

https://gerrit.wikimedia.org/r/965796

Change 965794 merged by jenkins-bot:

[integration/config@master] Zuul: archive mediawiki/libs/metrics-platform

https://gerrit.wikimedia.org/r/965794

Change 965796 merged by jenkins-bot:

[integration/config@master] Remove metrics-platform jobs

https://gerrit.wikimedia.org/r/965796

Does that sound like the right path @phuedx ? Would I be stepping on your toes?

We should also mirror the project to GitHub so that we don't break updating the mediawiki-vendor repository (per T344733#9192434). With that done, I think that those steps are correct.

The GitHub repo is currently mirroring, see https://github.com/wikimedia/mediawiki-libs-metrics-platform/tree/main

Changing the default base branch under "Settings" for that repo for master ā†’ main should mean tha mediawiki-vendor pulls in the latest changes. Doing that late on a Friday feels like it could lead to disruptionā€”let's sync on this Monday and then we'll close this out :)

Does that sound like the right path @phuedx ? Would I be stepping on your toes?

We should also mirror the project to GitHub so that we don't break updating the mediawiki-vendor repository (per T344733#9192434). With that done, I think that those steps are correct.

The GitHub repo is currently mirroring, see https://github.com/wikimedia/mediawiki-libs-metrics-platform/tree/main

Changing the default base branch under "Settings" for that repo for master ā†’ main should mean tha mediawiki-vendor pulls in the latest changes. Doing that late on a Friday feels like it could lead to disruptionā€”let's sync on this Monday and then we'll close this out :)

  • Repoint the GitHub repo from master ā†’ main

Done now!

And mediawiki/vendor update is clean after the switch.

Thanks @phuedx for the pairing/assistance/reassurance :).

Calling this one complete!

Change 958901 merged by jenkins-bot:

[mediawiki/extensions/EventLogging@master] modules/lib: Use GitLab URL for lib/metrics-platform

https://gerrit.wikimedia.org/r/958901