Page MenuHomePhabricator

Gerrit does not update Cards in mediawiki/extensions (it is thus stuck on Beta Cluster)
Closed, ResolvedPublic

Description

A visit to http://en.m.wikipedia.beta.wmflabs.org/wiki/Special:Version shows Cards extension at 0.3.0 (194bf33) despite 0.4.0 being the current patch on master.

Something funky is going on...

Event Timeline

Jdlrobson raised the priority of this task from to Needs Triage.
Jdlrobson updated the task description. (Show Details)
Jdlrobson added subscribers: Jdlrobson, bd808.

It looks to me like the submodule in mediawiki/extensions.git isn't getting bumped for new merges. I don't know what black magic in Gerrit, et al controls that.

greg renamed this task from Cards stuck on version 0.3.0 on beta labs to Cards stuck on version 0.3.0 on Beta Cluster.Jan 29 2016, 10:34 PM

The black magic in gerrit is controlled by the git module configuration. If the submodule has "branch=" set to some branch, then gerrit updates the parent repo when the submodule repo has a commit on the matching branch.

Change 267394 had a related patch set uploaded (by Thcipriani):
Cards extension not updating

https://gerrit.wikimedia.org/r/267394

I manually bumped the Cards extension in mediawiki/extensions. I'm hoping that since the default_branch was changed in .gitreview (one of the commits in the submodule bump) that gerrit will straighten up and fly right.

Would be good to ask @hashar about this dark gerrit magic.

Change 267394 merged by Thcipriani:
Cards extension not updating

https://gerrit.wikimedia.org/r/267394

@thcipriani I can confirm that the version is bumped now. Is there any way I can check if it autoupdates on new patches without submitting a patch..? :) Is there anything left to do here to wrap this work up?

thcipriani claimed this task.

Is there any way I can check if it autoupdates on new patches without submitting a patch..? :)

This feels like a riddle :)

I'm honestly not sure. I know that gerrit is responsible for bumping submodules in the extensions repo, but my gerrit-admin-fu is somewhat weak in this area. @hashar is there a way to test this other than submitting a new patch?

Is there anything left to do here to wrap this work up?

At this time I can't think of anything, I'm closing this for the time being just to remove it from my todo list. If the issue persists, feel free to re-open.

As I understand it, but @demon might correct me:

We use a feature from Gerrit that let it magically update submodules in a given project.

The repository is mediawiki/extensions.git and in its master branch it has most extensions registered as submodule with a special parameter recognized by Gerrit: branch = ..

When the master branch of one of the registered submodule is updated, Gerrit magically bump the submodule pointer in mediawiki/extensions.git master branch. It has issues though VisualEditor/VisualEditor and mediawiki/extensions/VisualEditor have the same base name which confuses Gerrit :/

Looking on deployment-bastion:

Cards is at:

6583a51 (HEAD, origin/master, origin/HEAD) Merge "Design review"
CommitDate: Fri Jan 29 15:18:58 2016 +0000

And that is the tip of the mediawiki/extensions/Cards@master.

It is apparently a recurring issue and it seems it never properly updates:

# Fields: sha1 | commiter date | subject | author
$ TZ=C git log  --pretty=format:'%h %cd %s <%an>' --abbrev-commit --date=local Cards
0b60815 Fri Jan 29 23:53:48 2016 Cards extension not updating <Tyler Cipriani>
2b89a0e Sat Jan 16 20:32:48 2016 Cards is lagging <Reedy>
f8b4435 Fri Dec 18 13:04:55 2015 Cards is lagging <Reedy>
$
hashar renamed this task from Cards stuck on version 0.3.0 on Beta Cluster to Gerrit does not update Cards in mediawiki/extensions (it is thus stuck on Beta Cluster).Feb 4 2016, 1:55 PM
hashar reopened this task as Open.
hashar removed thcipriani as the assignee of this task.
hashar added a project: Gerrit.

Ah easy enough. There are two repositories named Cards:

Cards
mediawiki/extensions/Cards

So that is the exact same issue as VisualEditor which got tracked in T51846. The way we worked around it is that on postmerge of VE, we trigger the Jenkins job mwext-VisualEditor-sync-gerrit which takes care of doing the submodule bump.

It is an horrible hack and I would rather NOT disseminate it to Cards.

Since Cards.git is empty (a clone shows nothing, no review pending), I would recommend to delete Cards project from Gerrit entirely. Then that would stop confusing Gerrit hopefully magically.

But maybe we will need to do some magic SQL queries in Gerrit to fix the registration has shown on T51846#586113 (from 2013 so Gerrit schema might have changed).

SELECT * FROM submodule_subscriptions WHERE submodule_project_name LIKE '%Cards' AND submodule_project_name NOT LIKE '%TwitterCards';

submodule_project_namesubmodule_branch_namesuper_project_project_namesuper_project_branch_namesubmodule_path
Cardsrefs/heads/mastermediawiki/extensionsrefs/heads/masterCards
Cardsrefs/heads/wmf/1.27.0-wmf.10mediawiki/corerefs/heads/wmf/1.27.0-wmf.10extensions/Cards
Cardsrefs/heads/wmf/1.27.0-wmf.11mediawiki/corerefs/heads/wmf/1.27.0-wmf.11extensions/Cards
Cardsrefs/heads/wmf/1.27.0-wmf.11mediawiki/corerefs/heads/wmf/1.27.0-wmf.11.nosessionmanagerextensions/Cards
Cardsrefs/heads/wmf/1.27.0-wmf.12mediawiki/corerefs/heads/wmf/1.27.0-wmf.12extensions/Cards
Cardsrefs/heads/wmf/1.27.0-wmf.8mediawiki/corerefs/heads/wmf/1.27.0-wmf.7extensions/Cards
Cardsrefs/heads/wmf/1.27.0-wmf.8mediawiki/corerefs/heads/wmf/1.27.0-wmf.8extensions/Cards
Cardsrefs/heads/wmf/1.27.0-wmf.9mediawiki/corerefs/heads/wmf/1.27.0-wmf.9extensions/Cards

For mediawiki/core , the .gitmodules do not have branch = . so no magic update. The only line that matters is the first one. submodule_path reads as Cards, should be extensions/Cards. So it is all confused :-}

For VisualEditor the breakage is:

submodule_project_namesubmodule_branch_namesuper_project_project_namesuper_project_branch_namesubmodule_path
VisualEditorrefs/heads/mastermediawiki/extensionsrefs/heads/masterVisualEditor

From https://www.mediawiki.org/wiki/Git/New_repositories/Requests

/Cards.gitbmansurovMerge review

The Reading Web team would like to convert the mediawiki/extensions/Cards repo to a library. I think we can do the cloning ourselves, we just need the mediawiki/extensions/Cards repo to be deleted after this one has been created.

  • bmansurov

Done
Only Jenkins can Submit to the repo for now. If you want to be able to Submit yourself, you can grant it to yourself, or just let me know, and I'll happily grant it to you. Once you're done with cloning, let me know, and and we'll see to deleting the old extension.
--QChrisNonWMF (talk) 18:37, 15 November 2015 (UTC)

I ran update submodule_subscriptions set submodule_project_name = 'mediawiki/extensions/Cards' where submodule_path = 'Cards'; and update submodule_subscriptions set submodule_project_name = 'mediawiki/extensions/Cards' where submodule_path = 'extensions/Cards'; in gsql and then deleteproject delete --yes-really-delete -- Cards.git - just need someone to delete it from git.wikimedia.org now
Has that fixed things for mediawiki/extensions/Cards?

I ran update submodule_subscriptions set submodule_project_name = 'mediawiki/extensions/Cards' where submodule_path = 'Cards'; and update submodule_subscriptions set submodule_project_name = 'mediawiki/extensions/Cards' where submodule_path = 'extensions/Cards'; in gsql and then deleteproject delete --yes-really-delete -- Cards.git - just need someone to delete it from git.wikimedia.org now
Has that fixed things for mediawiki/extensions/Cards?

You do realize that won't fix anything and I've done the exact same thing for VE before right? It'll re-break itself.

You do realize that won't fix anything and I've done the exact same thing for VE before right? It'll re-break itself.

You deleted VisualEditor.git?

Change 268394 had a related patch set uploaded (by Alex Monk):
Try updating Cards extension

https://gerrit.wikimedia.org/r/268394

Change 268394 merged by jenkins-bot:
Try updating Cards extension

https://gerrit.wikimedia.org/r/268394

Krenair claimed this task.

that did the trick

New status:

SELECT * FROM submodule_subscriptions WHERE submodule_project_name LIKE '%Cards' AND submodule_project_name NOT LIKE '%TwitterCards';

submodule_project_namesubmodule_branch_namesuper_project_project_namesuper_project_branch_namesubmodule_path
mediawiki/extensions/Cardsrefs/heads/mastermediawiki/extensionsrefs/heads/masterCards
mediawiki/extensions/Cardsrefs/heads/wmf/1.27.0-wmf.10mediawiki/corerefs/heads/wmf/1.27.0-wmf.10extensions/Cards
mediawiki/extensions/Cardsrefs/heads/wmf/1.27.0-wmf.11mediawiki/corerefs/heads/wmf/1.27.0-wmf.11extensions/Cards
mediawiki/extensions/Cardsrefs/heads/wmf/1.27.0-wmf.11mediawiki/corerefs/heads/wmf/1.27.0-wmf.11.nosessionmanagerextensions/Cards
mediawiki/extensions/Cardsrefs/heads/wmf/1.27.0-wmf.12mediawiki/corerefs/heads/wmf/1.27.0-wmf.12extensions/Cards
mediawiki/extensions/Cardsrefs/heads/wmf/1.27.0-wmf.8mediawiki/corerefs/heads/wmf/1.27.0-wmf.7extensions/Cards
mediawiki/extensions/Cardsrefs/heads/wmf/1.27.0-wmf.8mediawiki/corerefs/heads/wmf/1.27.0-wmf.8extensions/Cards
mediawiki/extensions/Cardsrefs/heads/wmf/1.27.0-wmf.9mediawiki/corerefs/heads/wmf/1.27.0-wmf.9extensions/Cards

That is not appearing in this task history, but the Cards.git repository has been deleted.

Sounds like T70271: Upgrade gerrit to 2.12 will fix it for us

That puzzled me!

Upstream commented on our issue #2393 with:

Fixed in https://gerrit-review.googlesource.com/#/c/69891/
(which is in 2.11 or later)

Commit message being:

Subscribe to proper project when nested projects exist

Submodule subscription would parse from the end of the url and return a
subscription to the first project it would find. Now, continue to parse
the url and subscribe to the project found with the most complete name.

Example:
Project "a/b" exists
Project "b" exists
Submodule subscription url = ../a/b

Old behaviour:
Would subscribe to project "b"

Expected:
Subscription to project "a/b"