Page MenuHomePhabricator

User-level gadget repositories
Open, MediumPublic

Description

From @Nux's comment on T29488#313444

I've submitted this bug (T29488) with specific needs in mind. There is no way I know to:

  1. Load user scripts in the header to allow actually running their code BEFORE the page starts to render (needed to inject CSS or faster adding of crucial elements).
  2. Load user scripts in order (without the need of contacting administrators).

I'm not sure, but I think this probably means we need to provide a way to set position = top on user scripts. Maybe it is related to the need of creating modules from wikipages, maybe not.

See also:

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 12:11 AM
bzimport set Reference to bz34958.
bzimport added a subscriber: Unknown Object (MLST).

See also bug 27488 comment #68.

I suggest to rename this bug to "Allow creation of ResourceLoader wiki-modules on a user level", which will give a user script all ResourceLoader features (including the queue 'position', which is a ResourceLoader feature and should/can only by used by ResourceLoader modules).

Coming up with some kind of syntax to load imported user scripts from the top is not realistic in my opinion (nor practical).

Hrm, now I'm getting cranky because I think this is a dupe of bug 27488. Avoiding the dupe hammer for the moment, though.

a.d.bergi wrote:

(In reply to comment #0)

I'm not sure, but I think this probably means we need to provide a way to set
"position = top" on user scripts. Maybe it is related to the need of creating
modules fron wikipages (see bug 27281, bug 27561 and bug 25845), maybe not.

I think we should not provide a hacky way for setting "position=top". We really should go and implement wikipage-modules. I'm just not sure to which of these three bugs this here should be marked as duplicate.

None of those 3 are dupes. We now got:

  • bug 34958: Support for user-level resourceloader modules
  • bug 27488: Support for user scripts to have use load position (this will be solved naturally by 34958, but not marking as a dupe of it, bug 34958 is just one way of solving bug 27488).

These are currently filed under the MediaWiki core ResourceLoader component. That's fine for now but note that

  • User scripts are currently not loaded by ResourceLoader, they are "just" wiki pages loaded as raw scripts using plain HTML and JavaScript, nothing "MediaWiki"-ish about it. There is technically speaking no such feature as "user scripts" in MediaWiki.
  • Gadgets 3.0 will probably implement a feature that would replace (or introduce, depending on how you look at it) user scripts by allowing user-level repositories using JSON in the user space. At that point we could move these bugs to there, but lets keep it here for now.

What about allowing the user to define modules at [[Special:MyPage/modules]], e.g.:

User:Foo/modules

  • someModuleName
    • [[User:Foo/someModuleScript.js]]
    • [[User:Foo/moreModuleScript.js]]
    • [[User:Foo/someModuleStyle.css]]
    • [[User:Foo/moreModuleStyle.css]]
  • anotherModuleName
    • [[User:Foo/foo.js]]
    • [[User:Foo/bar.css]]
    • jquery.cookie (these are dependencies)
    • mediawiki.api
    • someModuleName
  • ...

What format it will be in is not a problem. The major barrier is the infrastructure behind it. Having a local startup module, controlling cache properly, clashing module names, foreign sources etc.

The format you propose is not realistic because it doesn't make sense to invent a new format. We already solved this problem once in Gadgets with Gadgets-definition syntax. Which we then destroy in Gadgets 2.0 because it doesn't scale enough.

The format will be JSON because wikitext does not scale for this kind of thing (we need scripts, styles, dependencies, messages, load position, and possibly more: skinStyles, skinScripts etc.). We already solved the format problem in Gadgets 2.0.

To update, user scripts are now loaded by ResourceLoaderUserModule. However, it's still just loading a fixed bundle of scripts (e.g. vector.js and common.js, depending on skin), which is quite different than allowing users to delineate modules.

Krinkle renamed this task from ResourceLoader: Allow creation of wiki-modules on a user level to Implement ability to create wiki-modules at a user level.Nov 25 2014, 8:47 PM
Krinkle updated the task description. (Show Details)
Krinkle removed a project: Future-Release.
Krinkle set Security to None.
Krinkle removed a subscriber: Unknown Object (MLST).
He7d3r added a subscriber: Nux.
He7d3r renamed this task from Implement ability to create wiki-modules at a user level to Gadgets 3.0: Implement ability to create wiki-modules at a user level.Sep 19 2015, 1:11 PM
Krinkle renamed this task from Gadgets 3.0: Implement ability to create wiki-modules at a user level to User-level gadget repositories.Feb 14 2019, 4:47 PM
Krinkle updated the task description. (Show Details)
Krinkle updated the task description. (Show Details)
Krinkle added a project: Proposal.
SD0001 added a project: Performance-Team.
Implementation proposal: user gadgets
  1. User-script authors define modules in their userspace, with page title having a .gadget suffix. These pages get the GadgetDefinition content model. Definitions are in the same format as the existing Gadget definition namespace (with some changes like not having a "default" and "hidden" keys).
  2. Users can save their list of enabled user modules (created by themselves or by others) via a titlesmultiselect field in gadgets tab of Special:Preferences.
  3. ResourceLoader modules are dynamically registered based on the current user. For contexts like load.php where there is no concept of a "current user", no user gadgets are registered.
  4. Dependencies and peers between user gadgets can be expressed using the titles of the definition pages (instead of the usual RL module ids).
  5. The above setup means that a user gadget is loaded only if you enable it from Preferences or if it's a dependency/peer of another enabled user gadget. Adhoc loading via mw.loader.load/using is NOT possible (but of course the individual scripts/stylesheets can be loaded that way). edit: It would be possible, see T36958#7643239
Things to think about
  1. Should the definitions of enabled user gadgets be cached (like site gadgets in WANObjectCache)? If so, how? (Tagging Performance-Team)
  2. Should the .gadget suffix be localisable? Admins likely wouldn't be given the permission to edit these definition pages, but they can change the suffix by editing the interface message. This doesn't really mean they can indirectly edit the definitions (since protection would be based on the content model, and changing the suffix only means new pages with the old suffix won't get the content model or protection – and they also wouldn't be treated as user gadgets), but are there other potential issues?

Change 751183 had a related patch set uploaded (by SD0001; author: SD0001):

[mediawiki/extensions/Gadgets@master] [WIP] User level gadget repositories

https://gerrit.wikimedia.org/r/751183

A user defined GadgetDefinition sounds interesting.

However, the exercise has two viewpoints:

  1. I am defining my own userspace in my Gadgets-definition.
    1. There is a one and only global name (but perhaps localized??) as page identifier and just one Gadgets-definition per user. Similar to Special:MyPage/Common.css which has a globally unique meaning. Well, better no localization but Special:MyPage/Gadgets-definition for all gadgets.
    2. I can define bundles of resources, one gadget identifier will need a particular .js, a sub-.js, a .css resource.
    3. There might be a mixture of dependencies, some resources from my own user space and some from site gadgets.
    4. Special:Gadgets may be generated for this particular user name: Special:Gadgets/user:SD0001
    5. It might become necessary to apply for appropriate content model, but new pages should be equipped properly.
  2. I am using gadgets of other people, perhaps from a different wiki.
    1. Using the URL format?
    2. Resource bundles, aka gadgets, are accessed by a gadget ID which is currently defined by MediaWiki:Gadgets-definition and which will need to be prefixed by user: and user nick and slash.
    3. Specifying my favoured default loadings even from other users and one day from other wikis might be a challenge. However, Special:Preferences#mw-prefsection-gadgets might be extended by a further chapter with sections taken from myself.

Why not extend to mw.loader.load|using("user:SD0001/doit") since that is just another gadget ID, and it is well defined how to create such module?

  • No problem with cacheable modules. Cached compressed source code bundles are benefits from that infrastructure.

A major goal would be to keep as much as possible in line with the site gadgets infrastructure, and to reuse as much code as possible. This will make it easier for both developers and global code maintainers as well as users within wikis. Do not re-invent the wheel.

There is a one and only global name (but perhaps localized??) as page identifier and just one Gadgets-definition per user. Similar to Special:MyPage/Common.css which has a globally unique meaning. Well, better no localization but Special:MyPage/Gadgets-definition for all gadgets.

For user gadgets, we would not want to use MediaWiki:Gadgets-definition syntax – the gadget definition namespace and associated content model (which are in master, though not enabled in production) is preferable, since all site gadgets would one day be migrated to that. An additional advantage of this approach is that users would not have to add arcane syntax to a personal subpage – instead they get an intuitive GUI on Special:Preferences:

Screenshot 2022-01-04 at 11.05.04 PM.png (660×1 px, 443 KB)

I am using gadgets of other people, perhaps from a different wiki. A. Using the URL format?

Site gadgets also don't support loading anything from a different wiki. I've filed T298561 to try resolve that. Whatever is the solution reached could be extended to apply for user gadgets too.

Why not extend to mw.loader.load|using("user:SD0001/doit") since that is just another gadget ID, and it is well defined how to create such module?

For an RL module to be loaded via mw.loader.using/load, it needs to be registered server-side. That's not something that can be done by a JavaScript function. Unlike site gadgets (all of which are always registered even if not enabled), it's not possible to register every single user gadget as there could be thousands which would blow up the size of module manifest in the startup module. Triggering server-side registration on demand from client side is essentially T29561.

I am thinking in three levels, and I could imagine that all three will be available one day:

  • global
  • site
  • user myself

I would expect in Preferences that all three chapters are offered in future. However, I am not sure which century. The generation of entries in Preferences is identical for all three origins. However, global specifications will need translatable texts.

Global brief explanations and headlines would need to be multilingual for current user language. They shall advertise the promised global gadgets available in all WMF projects. Their gadget ID might be preceded by global: to avoid collisions with site ID.

Nobody will start to implement and maintain different systems and introduce a deviating definition language for users. The form derived of Special:MyPage/Gadgets-definition and presented on Special:Preferences is exactly the same as for site gadgets but a different chapter on the same page.

The current MediaWiki: location of MediaWiki:Gadgets-definition will be just shifted to its own namespace #2302 one day, together with description and headline text elements. The syntax and implementation will be kept, just page name and content model are subject for change. And resources are to reside in #2300 namespace.

Since I would expect gadget ID in mw.loader.load("enwiki:HotCat") it is obvious for RL how to resolve the request into a resource module from that wiki. All requests from that wiki might be collected and merged into one cacheable multi-module URL for that wiki. This is the same as for resources from current site.

A Gadgets-definition Editor tool may be offered by someone some day, which could make it easier to generate and modify lines in Gadgets-definition via interactive support: Select available resource pages from associated namespace via narrowing page selector drop box, tic various options on/off or use multi-select elements for some options. Link to every dependency page source even as redlink. Since site and user are sharing the same format it is the same approach for both interface editors and regular users.

So far I've been able to get user gadgets to work with group=user and group=private, but with two major limitations:

  1. Dependencies don't load. It appears that both group=user and group=private embed modules in the page rather than link them, and both don't support dependencies (or just behave weirdly with them, see T299288)
  2. If the docs are to be believed, in group=user caches aren't shared with other users – which is bad because a single user gadget is same for all users so caches should be shared. In group=private I think they aren't cached at all.

On trying to use 'site' or any other string as the group, everything falls apart – modules are now linked rather than embedded, so in the call to load.php?modules=startup the usergadgets must get registered. But as the session is not initialised in load.php requests, there's no way to know who the user is and hence no way to read their preferences.

So it appears some resourceloader changes would be required – specifically a new group for user gadgets in which: edit: not required, see T36958#7643239

  1. Modules are embedded rather than linked (to overcome the above difficulty). Are there any performance issues with embedded modules?
  2. Dependencies should load
  3. Caches are shared across users. And ideally changes trigger cache misses like in group=user

Hm... Hi there. I was quoted on top 🙂, so two things:

I'm not sure, but I think this probably means we need to provide a way to set position = top on user scripts. Maybe it is related to the need of creating modules from wikipages, maybe not.

I actually do not need that as much any more. I use Stylus (a browser extension) if I want to make some changes quick. Also browsers are much faster then 10 years ago so that flickering effect when css is added is not as much as of a problem as back then. So AFAIK that part alone is probably not a big deal any more.

But anyway you seem to be doing something different I guess?

User-script authors define modules in their userspace, with page title having a .gadget suffix. These pages get the GadgetDefinition content model. Definitions are in the same format as the existing Gadget definition namespace (with some changes like not having a "default" and "hidden" keys).

Would that simply mean something like a package definition? Like for Gulp or Webpack? Because IIRC Resource Loader is basically like a Webpack from before Webpack existed, right? If so maybe it would be worth to allow devs to use Webpack on a remote server and publish a script that is already built? It would then be off your hands. No need to worry about build tools, and caching too.

I might have missed the point of the user-level repositories though. Sorry if that is so.

@Nux This task is essentially about T36958#387742, though I believe setting position is no longer an explicit RL feature (any module with just styles load without flicker).

Integrating Webpack into MediaWiki to replace or supplement RL's build steps is a different thing altogether warranting a separate ticket. Even an already built script still needs RL for caching and efficient delivery alongside other modules.

Change 756169 had a related patch set uploaded (by SD0001; author: SD0001):

[mediawiki/core@master] resourceloader: Minor tweaks to support user-level gadgets implementation

https://gerrit.wikimedia.org/r/756169

After doing some more research, I figured out a better approach (see linked patch, ready for review) that does not involve embedding modules on the page. This approach enables user gadgets to exist as normal modules, triggered by a user-level startup module. The issues with the earlier approaches are resolved, and only trivial resourceloader changes are needed. Contrary to bullet 5 in T36958#7594392, user gadgets can be loaded adhoc via load.php?modules=ext.usergadget.p12345 (where the number is the pageid of the definition page) though this would not be considered officially supported.

With the above patch, preferences of enabled user gadgets are public. This helps with caching of the gadget startup module (group=user which enables them to be cached, unlike group=private which adds them to the HTML on every page load) – they get exposed from load.php?modules=ext.gadgets.userstartup&user=<username>.

I suppose this is fine regardless of the caching benefit as it mirrors the public nature of personal common.js pages, enabling technical editors to help others and identify use of malicious scripts.