I guess you'd have to add a directory => git URL mapping spec to the configuration file and add it to the dump in JsonCollector.

In T297384#7592808, @Tgr wrote:

I guess you'd have to add a directory => git URL mapping spec to the configuration file and add it to the dump in JsonCollector.

How does that help extensions not hosted on Gerrit?

The same way it helps extensions hosted on Gerrit - by being able to deduce a git URL from a directory name.

Alternatively, the script could just shell out to git and check the upstream repo URL for each directory. That would assume directories are git checkouts, but there isn't much point in doing otherwise. (Wikibase is a weird edge case with multiple extensions in the same repo, but I think it would work out in the end.)

In T297384#7593312, @Tgr wrote:

Alternatively, the script could just shell out to git and check the upstream repo URL for each directory. That would assume directories are git checkouts, but there isn't much point in doing otherwise. (Wikibase is a weird edge case with multiple extensions in the same repo, but I think it would work out in the end.)

I guess I should say I don't know how this script works.

From your description, It sounds like it is working through the a directory of extensions (from https://gerrit.wikimedia.org/r/mediawiki/extensions ??), so all one would need to do is to get the checkout of, say, https://github.com/MWStake/nonwmf-extensions/ and run it there.

Is that right?

A list of directories of extensions, yeah. The Toolforge config uses the two you mentioned.

Can we not examine the submodule remotes for the Git repo URL? I think that might work.

However we get the URL though, where should it be added? A new top-level repository key in each extension? (This will be the first thing to be added here that's not in extension.json.)

The other place that this info is sometimes stored is in composer.json (with key: support.source), but I suspect it's in not all that many extensions' files. We could replicate that naming though.

Change 751066 had a related patch set uploaded (by Samwilson; author: Samwilson):

[labs/tools/extjsonuploader@master] Add Git repository info to extension data

https://gerrit.wikimedia.org/r/751066

In T297384#7593476, @Samwilson wrote:

However we get the URL though, where should it be added? A new top-level repository key in each extension? (This will be the first thing to be added here that's not in extension.json.)

Thanks! this is exactly what I was asking for in this task!

I appreciate the help clarifying my request.

Actually, I think using the Git remote's URL isn't always correct. For example, CollapsibleVector's remote URL is https://gerrit.wikimedia.org/r/mediawiki/extensions/CollapsibleVector but in composer.json it's https://phabricator.wikimedia.org/diffusion/ECLV/ (although perhaps it shouldn't be pointing to Diffusion any more).

I've updated the above patch to first check the Composer value, and then fall back to the Git remote.

Change 751066 merged by jenkins-bot:

[labs/tools/extjsonuploader@master] Add Git repository info to extension data

https://gerrit.wikimedia.org/r/751066

Mentioned in SAL (#wikimedia-cloud) [2022-01-09T11:25:38Z] <wm-bot> <samwilson> Deploy fix for T297384 (upgrade from b98b3c20ca07b5d88c169056159b9619a5e21413 to 2908f0da80c997167e03a2eb6e4d3c29e28b4d16).

Oops, the new data has pushed us over the top:

Error when saving: The content you supplied exceeds the article size limit of 2048 kibibytes.

The JSON file at https://extjsonuploader.toolforge.org/ExtensionJson.json now contains the new repository field. But we'll have to remove some data, or move ahead with the splitting up of the Lua module, in order to get it working on-wiki.

I think there are a few fields that we could remove, at least for the time being. AutoloadNamespaces, MessagesDirs, and ResourceFileModulePaths, for instance, aren't used on-wiki anywhere (I think?). Removing those would probably get us down a bit. Any objections?

Change 752338 had a related patch set uploaded (by Samwilson; author: Samwilson):

[labs/tools/extjsonuploader@master] Remove more extension.json keys to reduce size

https://gerrit.wikimedia.org/r/752338

Change 752338 merged by jenkins-bot:

[labs/tools/extjsonuploader@master] Remove more extension.json keys to reduce size

https://gerrit.wikimedia.org/r/752338

Mentioned in SAL (#wikimedia-cloud) [2022-01-09T23:14:59Z] <wm-bot> <samwilson> Deployed 011c3a612dc69ce01a4ef9ef828a53374db0dcf0 T297384

Yep, it's working now:
https://www.mediawiki.org/w/index.php?title=Module:ExtensionJson&diff=next&oldid=5007869

I've updated the docs at https://www.mediawiki.org/wiki/Module:ExtensionJson/doc