Page MenuHomePhabricator

Allow a module to load (require) another module from a global module repository wiki
Closed, DeclinedPublic

Description

Recently I've been having conversations with various relevant people about creating global templates and modules. One proposal that came up from @daniel is to make a way to import a module that resides on a global wiki from a module that resides on a local wiki without allowing the invocation of a global module directly from a wiki page.

According to @daniel, this can be done without making a full implementation of general change propagation and dependency engine mechanisms (T201004). It will also create a feature that will be needed in any case for at least some global modules: version management, that is, making sure that a particular version of a module is used on a local wiki (it is needed for sensitive modules, upgrading which can break wikis).

This will not create a full implementation of global modules because it will require doing something, even if it is something easy on every wiki that wants to use a global module, and the whole point of global modules is that they will be immediately usable everywhere, like Commons images. This still will have to be done (T14610). Nevertheless, the importing from a global repository is a useful and probably necessary step in the right direction.

Event Timeline

Amire80 renamed this task from Allow a module to import another module from another wiki to Allow a module to import another module from a global module repository wiki.Jan 29 2020, 11:42 PM
Amire80 created this task.
Amire80 updated the task description. (Show Details)

According to @daniel, this can be done without making a full implementation of general change propagation and dependency engine mechanisms

Lacking that would mean edits to the "global" module wouldn't propagate to the local wikis. That seems like something that would quickly be reported as a bug.

It would probably also result in a lack of the ability to find other-wiki uses of the global module, which would also likely be quickly reported as a bug.

It will also create a feature that will be needed in any case for at least some global modules: version management, that is, making sure that a particular version of a module is used on a local wiki (it is needed for sensitive modules, upgrading which can break wikis).

That feature does not seem to be a part of this proposal. Rather than "allow a module to import another module", that would be more along the lines of "allow a module to import a specific revision of another module". This would be a somewhat unusual feature, as all other transclusion in MediaWiki uses the current (or FlaggedRevs stable) version of the target page. Allowing for transcluding old content from a specific revision seems like a change that would require more architectural-level consideration.

According to @daniel, this can be done without making a full implementation of general change propagation and dependency engine mechanisms

Lacking that would mean edits to the "global" module wouldn't propagate to the local wikis. That seems like something that would quickly be reported as a bug.

Yes, that's why it's just the first step—to allow some kind of propagation, of one revision at a time.

It would probably also result in a lack of the ability to find other-wiki uses of the global module, which would also likely be quickly reported as a bug.

Same here. This is a first step. My detailed global templates (and modules) proposal has a requirement for dependency tracking. Without implementing it, it's indeed not complete.

It will also create a feature that will be needed in any case for at least some global modules: version management, that is, making sure that a particular version of a module is used on a local wiki (it is needed for sensitive modules, upgrading which can break wikis).

That feature does not seem to be a part of this proposal. Rather than "allow a module to import another module", that would be more along the lines of "allow a module to import a specific revision of another module". This would be a somewhat unusual feature, as all other transclusion in MediaWiki uses the current (or FlaggedRevs stable) version of the target page. Allowing for transcluding old content from a specific revision seems like a change that would require more architectural-level consideration.

So, to clarify, that's @daniel's intention as I understand it: if you specifically make something that only imports (requires) a specific version, it will provide an opening, a "foot in the door" for proper cross-wiki module usage. Proper invocation and dependency can, and should come later.

And there is demand for a feature that imports only a specific version of a module. I heard such requests from editors, independently of my discussion with @daniel. Most of the time it shouldn't be needed, and the latest version of a module should be used, as it is on wikis now, but for some widely used or sensitive modules it can actually be useful.

Amire80 renamed this task from Allow a module to import another module from a global module repository wiki to Allow a module to load (require) another module from a global module repository wiki.Feb 16 2020, 10:14 PM

Being bold and adding the COVID-19 tag.

Why? Because as far as I could understand, it's a fairly easy thing to do, and it could help the wikis share common data much more easily.

For example, there is a table of COVID-19 cases that can be shared across wikis, but to actually share it, each wiki needs to write a module. And sure, people could copy these modules across wikis, but the data structure may change, the nature of presentation and modeling may change, and so on, and then the module will have to change, too. And it would have to change on every wiki.

Instead, you could make the bulk of the changes in the module on just one central module and let the local wikis focus on writing the prose in their languages, instead of wasting their time on rewriting the module.

If it's more difficult than what could be quickly done to address this problem, then whatever, remove this tag ¯\_(ツ)_/¯

Why? Because as far as I could understand, it's a fairly easy thing to do,

T122086: Use bot to share templates and modules between wikis might be a better way to go if you want this now, rather than rushing an incomplete implementation into Scribunto.

T122086#5106105 says that the bot called for already exists.

@Tgr to look into this (no longer than 1 day) and report back on possibilities

Can you load code through Data: tables and then pass it to loadstring?

Can you load code through Data: tables and then pass it to loadstring?

There's no loadstring in Scribunto (Extension:Scribunto/Lua reference manual#Removed functions and packages).

Implementation-wise this seems pretty straightforward (although I'm not familiar with the code base so I might be missing something): provide a variant of LuaEngine::loadPackage() that takes a remote title (and maybe a page revision), and add it to the package loaders. Package loaders always take a single module name as an argument, so it would have to take something like commons:Module:Foo or commons:Module:Foo|123456.

As others have already said, there would be no invalidation or usage tracking, which results in a number of problems:

  • If the central module is updated, local pages using won't be invalidated, so for example if the module is causing errors, after fixing it the pages with errors would have to be manually purged.
  • There will be no dependency tracking so things like "What links here" or "modules used on this page" won't show cross-wiki usage.
  • There will be no change tracking: if the central module is changed, it won't show up on the watchlists of people watching local pages which are affected by the change. (If module invocations need to be versioned this is not really an issue.)

The first two are handled for Commons images via the GlobalUsage extension, so that gives an idea of the complexity (it's about 2K lines of code). The third (injecting recentchanges entries for remote changes) is only done for Wikidata to date, I think.

After discussing this with @Abit I concur with what @Anomie has said above: T122086: Use bot to share templates and modules between wikis seems like a better approach, especially with the WMF trying to minimize deployment risk due to the ongoing pandemic. The necessary code change does seem simple, but Scribunto is a complicated extension few people are familiar with, and since the change affects wikitext parsing the potential fallout could be large. Also wikitext processing changes cannot be easily rolled back if they turn out to be not so useful. A bot is easier to both manage and dismantle.

Might be worth revisiting when we have more capacity for doing complex changes. In the meanwhile, let's make the next generation of dependency tracking (cf T201004: Spec out dependency engine interface, data structure, and states) supports cross-wiki features out of the box.

No problem.

The reason I created this task is that @daniel sounded keen to do it, and it sounded like a simple and reasonable intermediate step towards full implementation of global modules, so I wanted to document it.

I don't mind if it's closed as declined, as long as there is serious intention to implement T201004 and T41610 :)

Per @Tgr 's comment above, declining for now.