Page MenuHomePhabricator

Parser function for loading gadgets
Open, Needs TriagePublic

Description

A common subtype of community-maintained Javascript is one that adds interactive features to a certain template (examples include the Teahouse ask-a-question feature on enwiki, AddMe and FormWizard on meta, the tabbed interface used at mw:API:Login). These scripts are loaded and executed on every page and do some cheap test (such as checking whether some CSS class is present on the page) to determine whether to perform their main functionality. This is bad both for performance (all these scripts are loaded on all pages, even though they are only needed on a tiny fraction of pages, and usually ones that are only relevant for editors and as such never visited by most users of the wiki) and for robustness (bugs in these scripts would affect users viewing unrelated pages). Ideally scripts would only be loaded on pages where they are actually needed.

The infrastructure for that is mostly in place already - the Gadgets extension declares ResourceLoader modules for gadgets, gadgets can be marked as hidden (unavailable via the preferences page) and parser plugins such as tag extensions and parserfunctions can load ResourceLoader modules, so all that is needed is a parser function that does tell the parser to load the specified gadget, so it can be added to the relevant template. Something like {{#gadget:<gadget name>}} which would then load the ext.gadget.<gadget name> module.

Event Timeline

Another advantage would be the easy handling of making sure these gadgets run in edit preview mode (e.g. when testing template changes) which most of these gadgets get wrong.

I don’t think that this proposal should be implemented. It is a rather hard break in usual cooperation of interface, page content and JavaScript gadgets.

  • T204201 Extend MediaWiki:Gadgets-definition capabilities would solve the JavaScript loading issues much better. It serves much more cases and is flexible enough, e.g. by loading JavaScript if a certain template is transcluded.
  • Introduction of new <elements> will destroy the separation of HTML content, as provided by wikitext, and JavaScript which is part of current interface and basically independent of HTML. CSS is quite close to HTML presentation.
  • Gadget administration should be done in a central place, not scattered over source code of templates or even articles.
  • T204201 Extend MediaWiki:Gadgets-definition capabilities would solve the JavaScript loading issues much better. It serves much more cases and is flexible enough, e.g. by loading JavaScript if a certain template is transcluded.

But what if not the template is transcluded, only its sandbox? The module may or may not need to be loaded; this is not deterministic and known only to the—potentially non-IA—user testing in the sandbox. Or when one wants to preview a change to a template that adds or removes a gadget dependency? This latter one is not only organizationally, but also technically impossible to support with the architecture you propose.

  • Introduction of new <elements> will destroy the separation of HTML content, as provided by wikitext, and JavaScript which is part of current interface and basically independent of HTML. CSS is quite close to HTML presentation.

If we want to support certain templates with JavaScript, the JavaScript will depend on the page content no matter what. The templates present on the page are determined by the content, after all. Also, there are several extensions that already load JavaScript based on parser tags. Graph and Kartographer rely exclusively on this, but for example Scribunto also loads a JavaScript RL module to show script error details in popups.

  • Gadget administration should be done in a central place, not scattered over source code of templates or even articles.

Configuration of a certain template should be done in a central place, not scattered over the template page and MediaWiki:Gadgets-definition.

  • The proposal is also creating a risk of undiscovered vandalism.
    • Malicious users, even every anonymous user can trigger all available gadgets and increase client load of all visitors. Since not useful in current situation, no gadget will have any visible effect and attack may be detected years later.
  • It is not clear what is taken precedence over explicit user opt-out. Will the proposed element override user option?
  • There are not many templates which have a reasonable connection with a specific template.
    • A template for coordinates may trigger the local adaption of WISWOSM. That can be administrated by the new syntax of T204201 and narrow such gadget calls to pages where coordinates really occur.
    • If a gadget is related to a particular template it is up to inventors to tailor the usage precisely that transclusions and gadget loading are matching well.
    • It is even possible to define a dummy template which creates no wikitext output at all, and has the one and only purpose to trigger gadget loading. That is exactly what the proposed new element shall do, but limited to exactly those gadgets which are defined for such assignment.
  • There will be only very, very few templates which are related to a gadget.
    • The total number of gadgets is not very large, and it is an exception that a template is related to a possible gadget in that wiki.
    • There is not a huge number of applications with relation of content and JavaScript. Table sorting, collapsible elements. Those as well as extensions like Graph and Kartographer are under MediaWiki control.
  • Management of gadgets shall be kept limited to interface administrators by maintaining Gadget Definitions specifications.

This entire proposal has high risks of misuse. It has no benefit which would not be available via extended Gadget Definitions, but creates double work and complicated additional framework for a single exotic purpose.

Opt-out is a good point. I think the easiest way of handling it would be by having the parser tag add the gadget to the ParserOutput's extension data, and then that data could be filtered by user preference when the output is turned into a full page.

I don’t see my point about sandboxes (especially sandboxes edited by non-IAs) answered. Also,

  • The proposal is also creating a risk of undiscovered vandalism.
    • Malicious users, even every anonymous user can trigger all available gadgets and increase client load of all visitors. Since not useful in current situation, no gadget will have any visible effect and attack may be detected years later.

There are many ways malicious users can trigger load on users’ browsers. (I don’t want to help them by listing them here, but I have some ideas.) Also, if my proposal in T17075#6622308 about the restriction on gadgets available is implemented, only the few that are marked as available could be exploited. If some gadget requires a huge amount of code, it’s wise anyways to load as much of it as possible only upon user interaction.

  • It is not clear what is taken precedence over explicit user opt-out. Will the proposed element override user option?

Often such JavaScript is loaded by Common.js now, which neither allows opt-out. I don’t think it to be a huge concern.

  • There are not many templates which have a reasonable connection with a specific template.
    • A template for coordinates may trigger the local adaption of WISWOSM. That can be administrated by the new syntax of T204201 and narrow such gadget calls to pages where coordinates really occur.
    • If a gadget is related to a particular template it is up to inventors to tailor the usage precisely that transclusions and gadget loading are matching well.
    • It is even possible to define a dummy template which creates no wikitext output at all, and has the one and only purpose to trigger gadget loading. That is exactly what the proposed new element shall do, but limited to exactly those gadgets which are defined for such assignment.
  • There will be only very, very few templates which are related to a gadget.
    • The total number of gadgets is not very large, and it is an exception that a template is related to a possible gadget in that wiki.
    • There is not a huge number of applications with relation of content and JavaScript. Table sorting, collapsible elements. Those as well as extensions like Graph and Kartographer are under MediaWiki control.

How does the number of affected templates and gadgets matter? Yes, there won’t be many, so what? Granted, we could use a dummy template, but that’s just this proposal with a more obscure syntax, with all its advantages and drawbacks/risks.

  • Management of gadgets shall be kept limited to interface administrators by maintaining Gadget Definitions specifications.

Management mostly remains at IAs, the only exception being the decision what templates require which modules—which depends on the templates as well as the gadgets. If template authors cannot directly influence the loaded gadgets, they’ll likely find workarounds instead of having to hunt an IA, especially on smaller wikis, where there are only a handful of IAs, maybe none of them available at that moment.

All things being considered, I think a parser function for loading gadgets would be a net positive, and better than having gadget definitions based on templates because (per the comments here and at T204201):

  1. If the template is moved, the gadget loading behaviour should move with it
  2. The gadget should also load on template sandboxes to facilitate development

There are a number of occasions where communities have held off on creating a default gadget that interacts with certain content because it loads on all pages (even if it's just a 2-liner that checks whether the core gadget should load) – for example, ChessBrowser on en.wp.

But I don't think there's any usecase for embedding Twinkle or HotCat or Popups or CommentsInLocalTime on any page. Most of the gadgets that exist today probably aren't suitable to be invoked this way, least of all the ones that drastically change the appearance of the page like dark-mode or BlackSkin. A per-gadget opt-in (maybe an embeddable flag in gadget definition) would be a better choice than allowing user opt-out. In addition, we could even put a parser limit as we do for most parser features (say max 10 gadgets per page) if it's still required to prevent a vandal from using a large number of gadgets to increase client load.

Once we switch to the the gadget definition namespace, we can also make the parser function leave a backlink to the gadget definition page, so it can be seen via Special:WhatLinksHere which pages are embedding the gadget. (Until then an insource search can be used.)

Change 759295 had a related patch set uploaded (by SD0001; author: SD0001):

[mediawiki/extensions/Gadgets@master] Add parser tag for loading gadgets

https://gerrit.wikimedia.org/r/759295

I really do not want to have to monitor uses of this particular parser tag.

That said, I think this use case may also be satisfied today by the supportsUrlLoad functionality now; for example, all of the listed have a feasible implementation with supportsUrlLoad. ("To see the full game, click the link." or similar.)

I really do not want to have to monitor uses of this particular parser tag.

What kind of monitoring do you feel such a parser tag would necessitate?

I really do not want to have to monitor uses of this particular parser tag.

What kind of monitoring do you feel such a parser tag would necessitate?

The general "don't do that" kind. :) JavaScript is mostly the last frontier in things that wiki-citizens can do in arbitrary places. This may have some value, but I think it will come with "gosh do we really need that there"?

I'm pretty sure there's an ancient declined task for arbitrary JavaScript on a per page basis, but of course it was proposed as different in implementation than gadgets would be here (I think that was arbitrary JavaScript........).

The general "don't do that" kind. :) JavaScript is mostly the last frontier in things that wiki-citizens can do in arbitrary places. This may have some value, but I think it will come with "gosh do we really need that there"?

The intended use case would be to include it in templates which have some sort of interactive behavior (similar to TemplateStyles). I don't see why editors would use it in random inappropriate places, and why it would be problematic to deal with it if they did.

I'm pretty sure there's an ancient declined task for arbitrary JavaScript on a per page basis, but of course it was proposed as different in implementation than gadgets would be here (I think that was arbitrary JavaScript........).

It should only be possible to include code that was specifically intended for inclusion, sure.

I don't see why editors would use it in random inappropriate places, and why it would be problematic to deal with it if they did.

I sometimes see comments like this and wonder if the people who make them even edit. ;) I have spent a year now removing some very creative uses of various global CSS classes. Users will always find a way to do something you didn't expect (see also Hyrum's law).

That said, I remain in the position that on-click interaction with supportsUrlLoad already fits 95% of the use cases for arbitrary addition of gadgetry somewhere, a comment you did not respond to. Is it worth it to make changes on this dimension with that one in play? Vice versa, does it suggest that the other thing -- just added -- should be removed? Gadgets that load on-click could basically just not need the on-click with this change, so is it worth it to support the on-click version with this change in place?

Another but safe approach tailored for this desire has been specified in T204201 by either transcludes= or a bit more risky categories= condition available to interface admins only.

It is always to be considered that

  • by incident someone is causing JavaScript loading on many pages where it is not useful, or
  • malicious attackers could load a pile of pointless gadgets on all pages to slow down presentation for all readers.

Gadget execution for everybody must not be laid into the hands of every internet user on this planet.

That said, I remain in the position that on-click interaction with supportsUrlLoad already fits 95% of the use cases ...

It isn't just on-click, it's on-click + page-reload. Requiring a reload of the entire page is a pathetic user experience which frankly I haven't seen anywhere else. It's barely acceptable for project page workflows, let alone for content in mainspace. Plus there are the remaining 5% (though I'd argue it's more) use-cases that we need to solve.

CSS classes are a lot easier to abuse. Gadgets can easily bail out when invoked in an unexpected context with an if condition at the top. All uses would populate a hidden category, so can be easily tracked (not that doing so is urgent as loading is quite cheap due to RL optimisations – if execution isn't taking place, there isn't much to worry about.)

Another but safe approach tailored for this desire has been specified in T204201 by either transcludes= or a bit more risky categories= condition available to interface admins only.

Gadget execution for everybody must not be laid into the hands of every internet user on this planet.

supportsEmbed= condition is available to interface admins only. Those approaches aren't any safer. Anyone can transclude a template or add a category to any page.

malicious attackers could load a pile of pointless gadgets on all pages to slow down presentation for all readers.

The patch provided puts an upper limit on number of gadgets that can be loaded via the parser per page (default 10). On subsequent tag uses, an error message is emitted instead. (If we were to use transcludes= or categories= instead, this can't be cleanly done as no error message can be shown in response to template or category syntax.)

The transcludes= is much safer than any suggestion in this task or any other.

If a particular template needs support beyond wikitext, e.g. a sequence of images which can be flipped at the same layout position and shall react on clicks on forward or backward etc., this template can be connected with the JavaScript gadget. Any transclusion of such images is obvious and misusing will raise questions.

The proposed parser function can be executed silently somewhere deep in template hierarchy, requesting many gadgets with no reason. It might have been forgotten or could be an attack. It has no visible effect on readers rather than slowing down page rendering, loading and executing rubbish.

The same goes for categories which might be built in silently and could trigger bad resource loading.

Once we switch to the the gadget definition namespace, we can also make the parser function leave a backlink to the gadget definition page, so it can be seen via Special:WhatLinksHere which pages are embedding the gadget. (Until then an insource search can be used.)

Technically you could start adding the backlinks now given that the namespace already exists...might be confusing though. Maybe use a page property in the meantime?

I have spent a year now removing some very creative uses of various global CSS classes. Users will always find a way to do something you didn't expect (see also Hyrum's law).

TemplateStyles for example is far easier to misuse because anyone can create CSS "gadgets". Yet, it doesn't seem to be a problem in practice.

That said, I remain in the position that on-click interaction with supportsUrlLoad already fits 95% of the use cases for arbitrary addition of gadgetry somewhere, a comment you did not respond to.

It's possible to do that but it's poor UX (slow, position in document will be lost, painful to make work for more than one gadget on the page...).

Gadgets that load on-click could basically just not need the on-click with this change, so is it worth it to support the on-click version with this change in place?

There's certainly an overlap but URL-based loading can be used for non-content-related use cases, like customizing an edit form or enhancing a special page.

Another but safe approach tailored for this desire has been specified in T204201 by either transcludes= or a bit more risky categories= condition available to interface admins only.

That's harder to manage and doesn't seem to solve any of the problems you and @Izno are worried about - one could then just use transcluded template the same way they would use the parser function.

It is always to be considered that

  • by incident someone is causing JavaScript loading on many pages where it is not useful, or
  • malicious attackers could load a pile of pointless gadgets on all pages to slow down presentation for all readers.

These don't seem worse than other kinds of template vandalism (one could trigger loading lots of ResourceLoader modules by using all kinds of extension hooks in a template for example), and can be limited the same way, by protecting frequently-transcluded gadgets.

Found the ancient task that I may or may not have mentioned, it was T8883: Allow page-specific inclusion of <script>s, etc. in header. Thanks @Xaosflux for declining it today ;). That task's subs might be worth merging here, not sure.

Yes, this may provide a solution to the user story from T8883.

I was implementing what I though it was a clever idea: Load gadgets based on a category.

i.e. in JavaScript we already have mw.config.get( 'wgCategories' ) that returns the category where the page belongs to. So I decided to create/use a template for a gadget, that will add pages using it to that category. The category will be a hidden one. This also serves the purpose of tracking which pages will be using the gadget.

Then, in MediaWiki:Common.js, put:

if ( mw.loader.getState('ext.gadget.XXXXX') && mw.config.get( 'wgCategories', [] ).indexOf( 'Pages using gadget XXXXX' ) !== -1 ) {
	mw.loader.load('ext.gadget.XXXXX');
}

This is working fine. However, I was struggling why that wasn't working on mobile (this code was copied to Mobile.js as well). Well, how frustrating is it: T121791: Remove category entry from config in mobile skin

That goes back to the same issue of: having to run js on every view to see "am I on this gadget page" as well.

That goes back to the same issue of: having to run js on every view to see "am I on this gadget page" as well.

I didn't want to wait forever for a functionality that's not gonna happen. See T8883#6851218 for rationale. This JS is very cheap and way more flexible than the functionality being suggested here.

This comment was removed by Gustmd7410.

This also serves the purpose of tracking which pages will be using the gadget.

The linked patch automatically categorises pages embedding gadgets – serving the tracking purpose. Pages using <gadget name="foo"/> get the category Pages using gadget foo.

I don't think the comparison to TemplateStyles regarding security risk is fully apt. I think the worst that can happen with style rules is that the real destination of a link can be obscured. This can be a problem but the damage is constrained to things that can happen when you visit some arbitrary URL. Malicious Javascript, on the other hand, runs in the context of the client and can do anything the Javascript API allows.

Back in May, I asked Krinkle his opinion about this change and got very distracted with other things and never posted it here, sorry.

Krinkle was mildly opposed, that it is likely to make the ecosystem worse in mid/long term if we approach it this way.

<Krinkle> I've generally tried to push towards passive configuration for this. The only strong use case here is performance, which I think we can address better in other ways. Also worried about the mobility of content as result, the fragile link between templates and gadgets on the wiki, loading style modules instead of templatestyles (why?), and introducing quite easily the misunderstanding that JS is part of the page and not eg an optional enhancement,
<Krinkle> consider Kiwix, and all other content reuse. Passive configuration I think better models that reality and vision. Also: skin specific requirements, and user rights. What does it mean to include the gadget but it's not for this user or skin etc.
<Krinkle> Namespace/action/category is the current proposal on the task for this. (T63007: Allow specifying when a gadget should load (action, namespace, content model))
<Krinkle> On a technical level , it's fine. But I'd like to see better rationale and/or a POV describing these concerns are not likely an issue.

Malicious Javascript, on the other hand, runs in the context of the client and can do anything the Javascript API allows.

Gadgets can only be added by trusted users, and if that gets circumvented, it makes little difference whether the attacker enables them on all pages or specific pages. If there will be differentiation between the rights needed for adding opt-in gadgets and default-on gadgets, it will make sense to treat the supportsEmbed flag as equivalent to default-on, but currently there's no such differentiation.

<Krinkle> I've generally tried to push towards passive configuration for this. The only strong use case here is performance, which I think we can address better in other ways. Also worried about the mobility of content as result, the fragile link between templates and gadgets on the wiki, (...) and introducing quite easily the misunderstanding that JS is part of the page and not eg an optional enhancement,
<Krinkle> consider Kiwix, and all other content reuse. Passive configuration I think better models that reality and vision.

I'm not sure that's reflective of current web practices - modern frontend frameworks do typically offer pseudo-tags which bundle HTML, CSS and JS into one conceptual unit, and that's also how our current OOUI widget library works on the server side. Forcing the community to enumerate uses of a gadget in an interface-admin-only registry is a very contrived and user-hostile way of forcing people to think about accessibility and graceful degradation. Reusers like Kiwix won't be affected if they only want the no-JS version and will probably benefit slightly if they want to include or reimplement the JS.

The strongest case here isn't performance, it's that this is the natural way to manage content-specific Javascript. For the same reason we don't use a central registry to determine e.g. which special page should load which JS module, we shouldn't do that for content templates either.

loading style modules instead of templatestyles (why?)

Why not? We could disallow making style modules embeddable, if we wanted, but I'm not sure I see the point. They are more restricted permission-wise but allow for more modern CSS.

Also: skin specific requirements, and user rights. What does it mean to include the gadget but it's not for this user or skin etc.

That it does not load. But it seems unlikely that a gadget would be both page-specific and userright- or skin-specific (other than providing different implementations for different skins, maybe) – the use case here is enhancing template content, and content itself isn't skin- or privilege-specific. (Well, usually not. Minerva hides all kinds of things and some wikis CSS-hide admin-only links and such. But it's not trouble to support that with the parser function.)

<Krinkle> Namespace/action/category is the current proposal on the task for this. (T63007: Allow specifying when a gadget should load (action, namespace, content model))

That's an orthogonal proposal that's useful for different types of gadgets (namespace/action, anyway; I'm not sure category is useful), and wouldn't at all help with this use case.
Binding gadgets to templates (T17075: Per book, category and/or template CSS and JavaScript) via the gadget registry would, but I don't think that avoids any of the supposed disadvantages of a parser tag, and is less convenient (e.g. how would you do backlinks?).

<Krinkle> ... The only strong use case here is performance, which I think we can address better in other ways.

The use case isn't performance. For the better or worse, communities have been unwilling to use default gadgets or sitewide common.js snippets to enable functionality on a limited set of pages. So this is an enabling feature rather than one which merely improves performance.

<Krinkle> Also worried about the mobility of content as result, the fragile link between templates and gadgets on the wiki ... On a technical level , it's fine. But I'd like to see better rationale and/or a POV describing these concerns are not likely an issue.

Whether it harms the ecosystem depends on how the feature is used, which is up to the communities to regulate. The enwiki manual of style stipulates that even colours and collapsed boxes should not be used to communicate essential information in an article, for accessibility. I'm sure a similar policy will be adopted for JS content, so that all page content remains accessible to screen readers, printers, kiwix, alexa, etc.

loading style modules instead of templatestyles (why?)

Why not? We could disallow making style modules embeddable, if we wanted, but I'm not sure I see the point. They are more restricted permission-wise but allow for more modern CSS.

Since current TemplateStyles implementation breaks in some middlingly common contexts (cf. T200704), style modules would be a reasonable workaround for some use cases. Not indiscriminately, of course, but that's a best practice-type issue.

With the closing of T63007: Allow specifying when a gadget should load (action, namespace, content model), especially the patch rEGADe2cb01ddcd66: Add support for categories in definitions, I think the premise for this task is also fulfilled. This task has been about using a magic word to add gadgets only to certain pages, but you can achieve the same by adding the pages you want to a (probably hidden) category that is only added to the pages the gadget is needed on.

So perhaps this can be closed?

I agree, I think that satisfies the desire for this task. (I think I'd like to see "template used on page" to load a gadget so that we can avoid adding some inevitable categories that will duplicate template links 1:1, but that's a separate feature request.)

I don't think this should be closed just yet. This is still the "more correct" approach. The categories stuff is just a hack to avoid introducing new wikitext.

The parser tag adds the RL modules directly into the parser output – enabling any client which uses the parsing APIs (eg. Live preview, real-time preview, VisualEditor, mobile apps, Apple Dictionary, etc) to see the added modules. Whereas the category-based implementation uses the BeforePageDisplay hook which only runs on web views, and other clients would have to implement custom code specific to Gadgets.

Even in the standard (non-ajax) preview, whether or not the gadget is loaded depends on whether or not the category exists in the last saved version, not the version currently being previewed. nvm, this does get handled correctly due to the fake revision setup.

Also, when you associate a category with a gadget, it does nothing to purge the CDN caches of pages in the category, so it can take upto months before it loads for logged-out users.

There's also various creative possibilities from using the parser tag in interface messages - which would enable running gadgets on non-view actions and special pages, which is not possible with categories. A use case is T240647, where you would want to load the unit testing framework only while editing.

Not to mention the abuse of the link tables - using a category for loading WikiMiniAtlas in enwiki will add 1.3 million rows in categorylinks.

Most of those issues could probably be fixed by using the ContentAlterParserOutput hook instead of BeforePageDisplay, to detect the relevant categories and add the gadgets to the parser output.

The cache purge probably just requires queueing a links update job with the right parameters; I think that would be the same with a parser function (which would presumably be tracked in the templatelinks table).

The cache purge probably just requires queueing a links update job with the right parameters; I think that would be the same with a parser function (which would presumably be tracked in the templatelinks table).

I doubt if it would be trivial as we need to parse the diff of Gadgets-definition, figure out which categories are being added or removed, fetch lists of pages in those categories, enqueue links update for each.

With the parser tag, we just add an entry in templatelinks and get all of the heavy-lifting for free as provided by core.