Page MenuHomePhabricator

Move math related repositories from GitHub(?) to gerrit
Closed, ResolvedPublic

Description

I filed a report on https://www.mediawiki.org/wiki/Gerrit/New_repositories/Requests to move math-related repositories to Gerrit as @Pchelolo, who was recently doing code review for those repos, did not have access to the respective repos, despite global +3.

Moreover, @hashar @Jdforrester-WMF and others improved the internal CI tremendously, so that there are only disadvantages having the code on GitHub. (tendentious, non-objective statement)

I think it would be good to keep the original repo names on GitHub since Google found about 100 hard links to texvcjs in the wild alone. As I, myself will be able to find the things I am looking for despite broken links, I would appreciate if one could spend extra effort to keep the old links intact. However, this is no conditio sine qua non to me.

Event Timeline

I think it would be good to keep the original repo names on GitHub since Google found about 100 hard links to texvcjs in the wild alone. As I, myself will be able to find the things I am looking for despite broken links, I would appreciate if one could spend extra effort to keep the old links intact. However, this is no conditio sine qua non to me.

You can move and rename repos (even giving them to organisations AFAIK) and github seems to keep redirects in place

A simple example being https://github.com/wikimedia/mediawiki-core -> https://github.com/wikimedia/mediawiki

If the repos are pushed from github, the objects will be the same, so shouldn't need fully re-replicating, so we can rename in place

We'll probably need to move https://github.com/wikimedia/mathjax to https://github.com/wikimedia/services-mathjax and back again to get the redirect (and similar for the other repo), but it shouldn't be much work

Redirects exist for various MW libs too

or find another way to give Pchelolo access to the repos

I've just added him to the "math" team.. Which probably more than gives him more "access"...

https://github.com/orgs/wikimedia/teams/math/members

Physikerwelt moved this task from Backlog to ready to go on the Texvcjs board.

@Reedy thank you. The redirect approach is what I used for previous projects. This works to my experience perfectly fine, including deep-links. See for example https://github.com/physikerwelt/texvcinfo

I fully support using the official naming convention as the final name on GitHub.

Since @Pchelolo now has access I lower the priority to low.

@QChris can you create the links as @Reedy suggested?

Right now this task isn't actionable. Which repos? From where to where? Etc.

Jdforrester-WMF renamed this task from Move math related repositories to gerrit or find another way to give Pchelolo access to the repos. to Move math related repositories from GitHub(?) to gerrit.Apr 14 2020, 8:32 PM
Physikerwelt removed QChris as the assignee of this task.

@Jdforrester-WMF thank you for updating the task description. What is the next step?

Change 593500 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/services@refs/meta/config] releng to forge identities for mass import of repos

https://gerrit.wikimedia.org/r/593500

Change 593500 merged by Hashar:
[mediawiki/services@refs/meta/config] releng to forge identities for mass import of repos

https://gerrit.wikimedia.org/r/593500

Change 593507 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/services@refs/meta/config] Add push to releng for mass import

https://gerrit.wikimedia.org/r/593507

Change 593507 merged by Hashar:
[mediawiki/services@refs/meta/config] Add push to releng for mass import

https://gerrit.wikimedia.org/r/593507

I have mirrored all the repositories using git clone --mirror then git push gerrit --mirror. That keeps all references.

  • texvcinfo
  • texvcjs
  • mathjax-node
  • mathjax

The texvcinfo github repository has a refs/for/wip%topic=svgo reference that points to d321963. I can't push it to Gerrit since refs/for/* is reserved to propose a change and that commit lacks a Change-Id header. I have manually edited it to add a change id and send it for review: https://gerrit.wikimedia.org/r/#/c/mediawiki/services/texvcinfo/+/593508

The texvcjs repository is similar. Github has a refs/for/master%topic=svgo that points to 54e9412d8. But that is already in the master branch so I guess it is fine to ignore it.

I have compared the list of references for all repositories (using git ls-remote) and they seem fine.

I am still importing mathjax my internet connection today for some reason.

mathjax has been imported.

I have renamed all the repositories in GitHub to use the new names.

Then I guess we will want to enable CI on all those repositories using the new Pipeline thing. There are guides available at https://wikitech.wikimedia.org/wiki/PipelineLib/Guides

@hashar thank you very much

Maybe we will want to create a new common group for all of them in case we need to add specific owners to those projects?

There is also the mathoid group https://gerrit.wikimedia.org/r/#/admin/groups/697,members and the math group https://gerrit.wikimedia.org/r/#/admin/groups/448,members maybe we can just use the math group for all of those. Ofter a functional change requires changes in more than one of those repos.

Change 593560 had a related patch set uploaded (by Physikerwelt; owner: Physikerwelt):
[mediawiki/services/texvcinfo@master] Add minimal testing for texvcinfo

https://gerrit.wikimedia.org/r/593560

Change 593563 had a related patch set uploaded (by Physikerwelt; owner: Physikerwelt):
[integration/config@master] Enable tests for texvcinfo

https://gerrit.wikimedia.org/r/593563

I was trying to follow Setup CI https://wikitech.wikimedia.org/wiki/PipelineLib/Guides but I am not sure how to test if what I did makes sense at all. I could run blubber locally but "check experimental" did not create a job in zuul.

Change 593563 merged by jenkins-bot:
[integration/config@master] Enable tests for texvcinfo

https://gerrit.wikimedia.org/r/593563

Mentioned in SAL (#wikimedia-releng) [2020-04-30T18:32:22Z] <James_F> Zuul: Install CI for mediawiki/services/texvcinfo T249908

Change 593560 abandoned by Jforrester:
Add minimal testing for texvcinfo

https://gerrit.wikimedia.org/r/593560

@hashar thank you very much

Maybe we will want to create a new common group for all of them in case we need to add specific owners to those projects?

There is also the mathoid group https://gerrit.wikimedia.org/r/#/admin/groups/697,members and the math group https://gerrit.wikimedia.org/r/#/admin/groups/448,members maybe we can just use the math group for all of those. Ofter a functional change requires changes in more than one of those repos.

There is also one for the MathSearch extension ( https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/extensions/MathSearch ).

I have created a math group https://gerrit.wikimedia.org/r/#/admin/groups/1776,members which you are now a member of. The group is owned by itself so you can add anyone to it and they in turn will be able to add other persons. I have made that group the owner of all the repositories I have ported from github as well as for mathoid service, the Math and MathSearch extensions.

https://gerrit.wikimedia.org/r/#/admin/groups/1776,members so if anyone is in maintaining those repositories, it should be all about adding them to that single group !

One more question on the texvcinfo change https://gerrit.wikimedia.org/r/#/c/mediawiki/services/texvcinfo/+/593560/ , it is mentioned the repository hosts a library, not a service. So I guess we should instead have created it as mediawiki/lib/texvcinfo ;D

@hashar Oups, I thought services would refer to the services team. In fact all of them are libs so to say.

I really love the PHP coverage tests, and I was wondering if it would be possible to use a different template such as

template:
  - name: node10-docker-coverage

which would also generate coverage tests for node-js?

@hashar Oups, I thought services would refer to the services team. In fact all of them are libs so to say.

I really love the PHP coverage tests, and I was wondering if it would be possible to use a different template such as

template:
  - name: node10-docker-coverage

which would also generate coverage tests for node-js?

To repeat what I said on gerrit last week when you asked, code coverage is the responsibility of your repo. See how it's done in VisualEditor or OOUI.

T116808 is the idea of doing it in CI, but it's unlikely we'll do it, sorry.

To repeat what I said on gerrit last week when you asked, code coverage is the responsibility of your repo. See how it's done in VisualEditor or OOUI.

T116808 is the idea of doing it in CI, but it's unlikely we'll do it, sorry.

@Jdforrester-WMF Thank you. I was looking at that, but I could not see how I could apply this for my repo in 5 minutes. And the coverage reports are also generated by my IDE and shown to the public on github via coveralls. Thus while this would be nice, the added value for only a few projects is too little.

Reopening this since I need some bit to be solved. The repository is apparently available in Gerrit and a change got recently merged there https://gerrit.wikimedia.org/r/c/mediawiki/services/mathjax-node/+/606742

However the Github repository is marked as archived https://github.com/wikimedia/mediawiki-services-mathjax-node and thus Gerrit is not able to replicate the change there.

Assuming the Gerrit repository is now the canonical one, I guess we can unarchive https://github.com/wikimedia/mediawiki-services-mathjax-node and update its description to state it is a mirror of https://gerrit.wikimedia.org/r/mediawiki/services/mathjax-node ?

Assuming the Gerrit repository is now the canonical one, I guess we can unarchive https://github.com/wikimedia/mediawiki-services-mathjax-node and update its description to state it is a mirror of https://gerrit.wikimedia.org/r/mediawiki/services/mathjax-node ?

yes

Thanks @Physikerwelt

And Gerrit has managed to replicate properly:

[2020-09-03 10:23:19,715]
Replication to git@github.com:wikimedia/mediawiki-services-mathjax-node completed
in 2985ms, 2641096754ms delay, 43273 retries [CONTEXT pushOneId="ed122988" ]

Same issue with the repository mediawiki/services/mathjax which Gerrit tries to replicate to https://github.com/wikimedia/mediawiki-services-mathjax though that one had a bunch of pull requests ( https://github.com/wikimedia/mediawiki-services-mathjax/pulls?q=is%3Apr+is%3Aclosed ). So probably we should rename it instead of deleting it.

I have renamed the legacy repo https://github.com/wikimedia/mediawiki-services-mathjax to https://github.com/wikimedia/mediawiki-services-mathjax-attic to make place for the Gerrit replicated one since we had:

Queue: ReplicateTo-github
1b5364cf 11:12:14.671 11:11:14.670      (retry 2524) [011fcb4d] push git@github.com:wikimedia/mediawiki-services-mathjax

To do so I had to unarchive the repository.

The item is out of the Gerrit replica queue so tentatively it is replicated though https://github.com/wikimedia/mediawiki-services-mathjax does not show up and redirect :|

I have renamed it back to https://github.com/wikimedia/mediawiki-services-mathjax

I think the replication was not working cause the repository was marked as archived. So it is fixed now.

I think this one is done. All the previous github repositories now redirect to the mirrored one. The Gerrit permissions have been set.