Page MenuHomePhabricator

Extend MediaWiki:Gadgets-definition capabilities
Closed, ResolvedPublic

Description

Please find some suggestions below.

  • The current concept of rights and skins and target etc. shall be extended.
  • The aim is similar to T63007 and perhaps in other places.
  • However, the syntax is extended within the current style.
  • Packaging shall be based on properties which can be evaluated immediately on loading.
  • The target is to avoid loading of meaningless modules, therefore each rule is further limiting the scope of gadgets required within the current page under current conditions.
  • In cache some more module packages will be founded per site, but repeating the same aggregation.
  • The rules still follow a simple AND combination. Some might wish an entire boolean language, but no implementation expected within this century.

Separators

Currently comma separation only is expected for multi-value.

  • Introduce space separator as regular syntax as well.
  • Keywords are simple ASCII words with no space inside, dependencies with dot, sometimes hyphen.
  • Tokenization is rather easy then, and quite usual.
  • Trimming between equal sign and pipe is no big deal.
  • Comma+space should not disturb anywhere.
  • Even may be used instead of space, or any whitespace.

That would improve readability.

Boolean considerations

  • If one keyword= rule is given, multiple items open an OR choice if not excluding.
  • All rules need to be matched, they form an AND expression.
  • A keyword= rule may occur multiple times (with other items). That is an AND junction of their items.
  • If items are prefixed with - this is an exclusion rule and all exclusions are to be met.

Namespace

Syntax:

namespaces=

Value is separated list of namespace numbers.

  • If element starting with - that might be exclusion rule.

Example:

|namespaces=14|

Load on category pages only. Otherwise the gadget will always start, first check namespace, abort immediately on most pages.

Special pages

Syntax:

specials=

Value is separated list of special page name keywords, case insensitive.

  • Keyword could be specialpages if desired, but only one of them.

Example:

|specials=Recentchanges,Watchlist,EditWatchlist|

Action

Syntax:

actions=

Value is separated list of &action= keywords.

  • edit shall cover submit as well.
  • If editing is not permitted to this user, view might be required and edit is bounced back (no editing tools meaningful).
  • If element starting with - that might be exclusion rule.
  • Some special pages are rewriting themselves into action=, e.g. Diff. That needs to be considered. It depends on the moment when the current situation is resolved.

Example:

|actions=view,edit,parsermigration-edit|

Do not load on history or info.

Content model

Syntax:

contentmodels=

Value is separated list of case insensitive content model names.

Example:

|contentmodels=css|

Run linter etc. on appropriate language only. Or limit to wikitext editing.

Page properties

Syntax:

pageprops=

Value is separated list of case insensitive pageprop names.

  • If element starting with - that might be exclusion rule (AND NOT).

Example:

|pageprops=templatedata|
|pageprops= -disambiguation -wikibase_item|
  • If TemplateData is present on this page, then link some keywords and add nice formatting.
  • If this article is not linked to WikiData and it is no disambiguation page then remind to look for WikiData item or create one if topic is appropriate.

Transcluded page

Syntax:

transcludes=

Value is whitespace separated list of page names with underscore _ rather than space.

  • If element starting with - that might be exclusion rule (AND NOT).

Example:

|transcludes=Template:citation_needed|

If that template is transcluded, load a gadget to find reliable sources.

Category

Syntax:

categories=

Value is whitespace separated list of category titles (no leading namspace Category:), with underscore _ rather than space.

  • If element starting with - that might be exclusion rule (AND NOT).

Alternative approach to transcludes= rule.

Example:

|categories=Wikipedia:Gadgets/Talk_utilities|

If current page is within that category, load a gadget to support discussions. Multiple templates might trigger such hidden category.

This approach has a certain risk to be misused for attacks, or editors are not aware of undesired effects triggered by categories. transcludes= is much safer.

User groups

Syntax:

groups=

Value is separated list of case insensitive user group names.

  • If element starting with - that might be exclusion rule (AND NOT).
  • all is not required but basic set anyway. Members may be subtracted.

Examples:

|groups=sysop|
|groups=-user|
|groups=user -sysop|

The second one is matching anonymous users. The third one are registered accounts, but might be not too experienced and powerful.

User language

Syntax:

userlangs=

Value is separated list of case insensitive language codes.

  • If element starting with - that might be exclusion rule (AND NOT).
  • Current user language pt-BR shall match |userlangs=pt|.

Example:

|userlangs=en-US,fr,zh-Hant,simple|

Shall exploit right-to-left scripting, translation helpers, special letter support, spellchecking etc.

Content language

Syntax:

contentlangs=

Same as userlangs but for page content language.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 624517 had a related patch set uploaded (by BrandonXLF; owner: BrandonXLF):
[mediawiki/extensions/Gadgets@master] Add support for namespaces in definintions

https://gerrit.wikimedia.org/r/624517

Change 747112 had a related patch set uploaded (by SD0001; author: SD0001):

[mediawiki/extensions/Gadgets@master] Allow specifying action pages in definitions

https://gerrit.wikimedia.org/r/747112

Change 747112 merged by jenkins-bot:

[mediawiki/extensions/Gadgets@master] Allow specifying page actions in definitions

https://gerrit.wikimedia.org/r/747112

I currently think, after page actions as conditions have been implemented, the next conditioning capabilities worthy of implementation can be special pages and categories. For example, in Russian Wikipedia:

  • We have as many as 7 default gadgets for special pages.
  • We also have a significant number of scripts/gadgets that run depending on the content of the page (generally, inside templates). I see "Transcluded page" is mentioned in the task as a separate condition, but I suspect a (most likely, hidden) category would serve better as a selector. Not too specific; template moves wouldn't affect gadgets; multiple templates can contain the same category.

There is also a huge choice of conditions described in this task that could be applied to many gadgets – say, a huge number of gadgets need to run only on contentmodels=wikitext, just like many of them could have actions=view, – but this categorization could turn out to be not particularly useful:

  • Non-default gadgets are not applicable to unregistered users. Unregistered users mostly visit actions=view + namespaces=0 + contentmodels=wikitext pages that is already a target for most default gadgets (those not requiring user rights), so no gain.
  • The overhead for registered users from specific gadgets being needlessly loaded to specific pages, I believe, is low. This is especially true for gadgets executed on the most common actions/contentmodels/etc. Except for Firefox users (see T235852), these gadgets already reside in the local storage. For Firefox users, they are cached. Probably the biggest overhead here is converting strings to bytecode (not sure).
  • Usually you can't even remove if ( mw.config.get( 'wgAction' ) === 'edit' ) or similar code from a gadget after adding a condition to Gadgets-definition because there are often cases when a gadget is loaded directly with mw.loader.load or something (and I often see @Jdlrobson adding dependencies to the code of gadgets after inspecting error logs, despite these dependencies are already specified in Gadgets-definition).

Until all of this gets ever implemented, it may be a good idea to split those gadgets into 2: Move all the gadget contents and dependencies to a new hidden gadget, and leave on the primary gadget only the logic for activation, and a call to mw.loader.load() that loads the hidden gadget. This would reduce the length of JavaScript code that gets loaded on all pages that don't make use of these. The drawback would be it takes more time to load (probably not critical for most gadgets).

As per template gadgets, it may be useful a magic word or TemplateStyles-like extension that loads a specific gadget on the page, rather than defining the list of pages/templates it should be loaded on the gadget definition. However, for security, those gadgets should probably be explicitly marked as "safe to be included in pages" in the gadget definition.

Until all of this gets ever implemented, it may be a good idea to split those gadgets into 2: Move all the gadget contents and dependencies to a new hidden gadget, and leave on the primary gadget only the logic for activation, and a call to mw.loader.load() that loads the hidden gadget. This would reduce the length of JavaScript code that gets loaded on all pages that don't make use of these. The drawback would be it takes more time to load (probably not critical for most gadgets).

We did exactly that (except that we didn't split each gadget – just included the conditioning code in Common.js) for default gadgets for actions, special pages and namespaces and now, as actions is introduced in Gadgets-definition, starting to remove this code. Hopefully the same happens for special pages soon. Namespaces could be less urgent/relevant.

As per template gadgets, it may be useful a magic word or TemplateStyles-like extension that loads a specific gadget on the page, rather than defining the list of pages/templates it should be loaded on the gadget definition. However, for security, those gadgets should probably be explicitly marked as "safe to be included in pages" in the gadget definition.

A list of pages/templates as a condition seems like a bad idea to me, I explained why. A more reasonable choice is a category that can be hidden. Hard to say whether a magic word/tag or an explicit option in gadgets definition is better. But not all gadgets that should load based on category could be described as "template gadgets". Both English Wikipedia and Russian Wikipedia currently have a universally loaded code for a BLP notice for pages with "Living people" and "Possibly living people" categories. In Russian Wikipedia, these categories are added by templates, but in English Wikipedia, they are placed in every article. So, a magic word will not work in that case.

A magic word/tag could also be seen as an element duplicating the functionality of a category. A category doesn't have to have the only function of activating a gadget on a page. It could also serve other purposes. If that's the case, and also if the category is placed in multiple templates, you would have to go and put a magic word/tag in all of them, which is not needed in case of an option in gadgets definition.

I often see @Jdlrobson adding dependencies to the code of gadgets after inspecting error logs, despite these dependencies are already specified in Gadgets-definition).

For this situation, I would approach it via a different bug. It would be useful if any attempt to load a gadget via action=raw was rejected by the host. Using gadgets without their definition shouldn't be possible as it creates work for gadget developers and undermines the meta in MediaWiki:Gadget-definitions.

e.g. https://en.wikipedia.org/w/index.php?action=raw&ctype=text/javascript&title=MediaWiki:Gadget-popups.js should not work.
mw.loader.using('ext.gadget.Navigation_popups') should always be preferred.

The gadget is composed of multiple .js .css .json resource files.

  • Some resource files may be shared between various site gadgets and user scripts, even from other wiki projects.
  • Every gadget resource should ensure that dependencies are satisfied, otherwise request now; and if conditions for namespaces, action, special page are necessary, they shall be checked again.
  • If dependencies are provided already per package, the RL will just confirm and continue; if missing for any reason they shall be loaded now. If why ever the code is executed within inappropriate context it should stop and might log an error message.
  • Gadgets-definition should provide efficient pre-loading of resources and dependencies by package, and start site gadget automatically only if meaningful. That is supported by this definition within local site.

Every useful gadget may be called from an external wiki. In this case, no Gadgets-definition is present, and there is no official support for cross-wiki usage of gadgets via URL yet.

  • Users might start gadgets not on preferences setting automatically when wikitext page is loading, but interactively on individual request when clicking a button, or in more restrictive user defined situations only, e.g. no other than articles.
  • Since no Gadgets-definition is in effect then, no code can be sure that it is never called rather than by site mechanism.
  • Good and robust gadgets always assert that dependencies and context match expectations, otherwise take suitable action, e.g. load missing modules now, or terminate if nothing to do. They simply work fine with no complaints ever under all conditions.

Every useful gadget may be called from an external wiki. In this case, no Gadgets-definition is present, and there is no official support for cross-wiki usage of gadgets via URL yet.

Wouldn't URLs like https://en.wikipedia.org/w/load.php?modules=ext.gadget.Navigation_popups work cross-wiki? (Although I agree, forbidding certain action=raw URLs seems a little extreme.)

I guess it will work, but I stated no official support for cross-wiki usage of gadgets via URL yet.

And please correct me if I am wrong, but from my experience load.php?modules= does not include or request dependencies nor messages. It is just the bundle of its own resources.

I cannot find those derived from Gadgets-definition, but fortunately popups.js does contain in line 7007:

// These dependencies should alse be enforced from the gadget,
// but not everyone loads this as a gadget, so double check
mw.loader.using( [ 'mediawiki.util',

BTW, you will need two URL calls, one for JavaScript and one for CSS.

I guess it will work, but I stated no official support for cross-wiki usage of gadgets via URL yet.

T262493#6455762 nicely summarizes this topic.

And please correct me if I am wrong, but from my experience load.php?modules= does not include or request dependencies nor messages

It includes messages, just not dependencies. (However, gadgets currently don't support including messages.)

BTW, you will need two URL calls, one for JavaScript and one for CSS.

No, unless you use &only=scripts or &only=styles filters, the same call includes JS, CSS and JSON (and messages).

BTW, you will need two URL calls, one for JavaScript and one for CSS.

No, unless you use &only=scripts or &only=styles filters, the same call includes JS, CSS and JSON (and messages).

Well, I am to retrieve for appropriate content model:

  • mw.loader.load( URL&only=scripts" );
  • mw.loader.load( URL&only=styles", "text/css" );

WRT to complaints about error log in T262493 when using load.php?modules=...&only=scripts

  • The result contains mw.loader.implement()
  • This one is screaming very loud if a module is touched which is already known.
  • It might happen easily that multiple branches will ask for the same module, e.g. local user common.js, then vector.js, previously user global .js, and perhaps a site gadget is loading it as well.

Always ask for mw.loader.getState().

var rls;
if ( ! mw.loader.getState( signature ) ) {
   rls = { };
   rls[ signature ] = "loading";
   mw.loader.state( rls );
   mw.loader.load( ...load.php?modules=ext.gadget." + signature + "&only=scripts" );
   mw.loader.load( ...load.php?modules=ext.gadget." + signature + "&only=styles", "text/css" );
}

…and if the local gadget has the same name as the remote one, you’re out of luck: mw.loader will always know the module, since you’re within the module with that name, and you can’t rename either gadget, as that would cause user preferences to be lost. (The &only=... solution will still work, though.)

BTW, you will need two URL calls, one for JavaScript and one for CSS.

No, unless you use &only=scripts or &only=styles filters, the same call includes JS, CSS and JSON (and messages).

Well, I am to retrieve for appropriate content model:

  • mw.loader.load( URL&only=scripts" );
  • mw.loader.load( URL&only=styles", "text/css" );

A simple load.php call includes everything, packaged as JavaScript (so the CSS is correctly fetched even though the request content-type is text/javascript).

Gadget IDs are to be registered globally; see T117540. If a local site is using the same ID for another purpose maximum confusion level is reached anyway.

The entire load.php?modules= is not official, derived from observation of current behaviour and most suspicious. All means to get robust results are to be used. Some hacks which might work today and fail tomorrow are not recommended. Therefore saving lines and robust statements since an undocumented feature might have been eavesdropped is not a smart strategy.

[…] Namespaces could be less urgent/relevant.

For Wikisource, and probably other sisters with multiple content namespaces, namespaces are a high priority. A large portion of our gadgets start by checking the namespace and bailing if in the wrong one. Second most common is probably checking wgAction (usually to see if we're editing or viewing, only exceptionally for diff etc.).

The namespace check is also an ugly lump of [ 0, 114 ].indexOf( ns ) !== -1 checks due to lack of ES6 support (which I don't think has landed yet, and comes with tradeoffs). When I can exile all that to MediaWiki:Gadgets-definition, and drop the two-.js-files-for-one-Gadget, I'll be unreasonably happy. :)

Change 922062 had a related patch set uploaded (by SD0001; author: SD0001):

[mediawiki/extensions/Gadgets@master] Add support for content models in definitions

https://gerrit.wikimedia.org/r/922062

[…] Namespaces could be less urgent/relevant.

For Wikisource, and probably other sisters with multiple content namespaces, namespaces are a high priority.

Hmm. And it occurs to me that we have a concrete use case for OR semantics: the enWP-derived Gadget-charinsert on enWS (which is not really related to Extension:Charinsert because of... reasons, I guess?) gets loaded anywhere on action=edit and additionally on Special:Upload. The reason is that the latter has a textfield for free text even in action=view.

The practical implication is that I can't move the action=edit check from the Gadget to MediaWiki:Gadgets-definition until Gadgets-definition supports 1) using special-page name as a condition, and 2) being able to specify OR semantics. Which means I can't get rid of the loader Gadget before that either.

enWP's equivalent Gadget has the same need, and without actually checking I'm guessing lots of projects either crossload or cut&paste enWP's code.

Change 624517 merged by jenkins-bot:

[mediawiki/extensions/Gadgets@master] Add support for namespaces in definitions

https://gerrit.wikimedia.org/r/624517

Change 922062 merged by jenkins-bot:

[mediawiki/extensions/Gadgets@master] Add support for content models in definitions

https://gerrit.wikimedia.org/r/922062

Change 1005092 had a related patch set uploaded (by Sophivorus; author: Sophivorus):

[mediawiki/extensions/Gadgets@master] Add support for categories in definitions

https://gerrit.wikimedia.org/r/1005092

Thanks for adding categories.

However, there is a serious syntax problerm: A category title may contain a comma. Imagine articles on Category:Castles (England, Wales).

A comma separated list is not appropriate. In the epic above I wrote:

Value is whitespace separated list of category titles (no leading namspace Category:), with underscore _ rather than space.

Change 1005092 merged by jenkins-bot:

[mediawiki/extensions/Gadgets@master] Add support for categories in definitions

https://gerrit.wikimedia.org/r/1005092

@Sophivorus @Krinkle can you clarify how this new |categories= syntax can be used with languages that can have comma in category names?.. Currently from looking at the code I see no way to do so, but category names like https://ru.wikipedia.org/wiki/Категория:Персоналии,_умершие_менее_года_назад are entirely correctly named in Russian. Asking for this use case:
https://ru.wikipedia.org/wiki/MediaWiki:Gadget-blpEditNotice.js

@stjn There is not currently a way to support categories that contain commas in their name. The "Gadgets-definition" parser uses the pipe |, equals =, and comma , symbols to parse tokens, and there is not currently any quotation or escaping mechanism in this parser. This applies to other options as well.

If we enable Gadgets 2.0 (T31272), this is automatically solved because the MediaWiki:Gadgets/id.json pages use JSON where categories are defined in an array like so. The JSON parser uses both quation and escaping, to allow pretty much any character to be represented.

{
  "module": {},
  "settings": {
    "categories": [ "Can, have, commas here!", "Something else" ]
  }
}

Other options would rarely contain categories, but this option is very likely to do so in many languages. I think making this change without considering this was bad.

The expected usage of this feature is with categories specifically created to be used in Gadgets-definition, so you can just not use a name that contains commas (until the Gadgets 2 migration happens).

This use case doesn't warrant using a gadget. A lua module can be used instead for generating the editnotice like we recently did in enwiki. Lua can read the page source to identify categories that indicate a BLP. If the category is added via a template, have lua search for the template instead. This will become more seamless and less hacky with T50175.

There is no specific template adding a category, it is added via heuristics. So no, it still requires a gadget while Lua support for categories is not there.

The expected usage of this feature is with categories specifically created to be used in Gadgets-definition, so you can just not use a name that contains commas (until the Gadgets 2 migration happens).

Not in every language that is a possibility (without butchering the names of categories), but OK, point taken.

I, on the other hand, would like to thank Krinkle and everyone else who made this humble patch happen, because even though it may not be perfect, it opens the door to so much useful magic. I started documenting some of it at https://www.mediawiki.org/wiki/Template_gadgets, and look forward to see where all of this will lead us. Cheers!

(I would like to amend the previous comment: I too consider the patch and the merge good for the project, I just think that given the constraints of the current system there should’ve been some amendment that would make possible to use category names with commas in them.)

Krinkle assigned this task to SD0001.

I'm closing this as resolved per T63007: Allow specifying when a gadget should load (action, namespace, content model).

The following are thus implicitly declined for now. However, feel free to create separate tasks for one or more of these (in that case, please provide an example gadget and wiki domain, and how the wikis' performance would benefit from it compared to alternative load conditions).

  • Individual special pages: premature optimisation, use namespaces=-1 instead.
  • User groups: we prefer user rights to improve portabilility across wikis, avoid or excluding new groups and global groups.
  • Transcluded, Page properties: use Content model or Categories instead.
  • User language, Content language: TBD, is there a use case? First concern would what the load hueristics are and whether it encourages large gadget registries.

Remembering that conditional loading is not where precise behavioural logic belongs. If we turn off conditional loading for any reason, there should be no difference in how a gadget behaves. The gadget must still verify and ensure what page it is on and whether it should do something. Conditional loading exists so that, for 99% of readers/editors that do not need the gadget, we try to avoid loading it. Loading it in a few rare cases where we don't need to, is not a problem.

If you use a gadget at least once during a one-day period of browsing, then it makes very little difference whether it is "queued" multiple times or only once, because we cache it inside the browser after downloading it the first time. The goal is that for people who do not need the gadget at all for an entire day, to not needlessly load it.

And... yes, that means conditional loading is primarily useful for default gadgets. For gadgets that are opt-in, it is more or less guruanteed that the person will likely download the gadget because they enabled it, presumbly with the intention to use it. Once it is downloaded, it will remain in the cache for two weeks or until the gadget is modified, after which "loading" it becomes very cheap.

About the conditional loading based on categories: If we edit a section (instead of the entire page) and the category wikicode is outside the section, will the gadget be loaded, or not?

Similarly, if the action if a history view, will the page categories be known?

I had a look at the code:

  1. The gadgets loader get the page categories from OutputPage::getCategories.
  2. This getCategories method gets the categories from the mCategories property.
  3. This mCategories property is filled by the addCategoryLinks method.
  4. This addCategoryLinks method is called either by setCategoryLinks or by addParserOutputMetadata. I haven't looked which one is called in the current case. Note the latter gets the categories from parser output (so on section editing or history view, I guess the categories would be unknown).

The category detection does not work when using quick preview and visual editor.
tested on fr:Moulinet_(échecs), Diaporama gadget is loaded in all cases but these two

About the conditional loading based on categories: If we edit a section (instead of the entire page) and the category wikicode is outside the section, will the gadget be loaded, or not?

If you mean during preview, yes it will load if the last saved revision of the page had the category wikicode. What section you're editing is irrelevant.

Similarly, if the action if a history view, will the page categories be known?

I believe not, but I don't think it should either.

The category detection does not work when using quick preview and visual editor.

Yes, it will not work in AJAX-based previews like Live preview, real-time preview, VisualEditor, mobile app previews, etc.


T241524 is a proposed alternative way to load gadgets which is free of these counter-intuitive behaviours.

Another issue: I have a case with a gadget ("Diaporama" here) that is defined with a "categories" condition, but this gadget may also be loaded using mw.loader.load()/ mw.loader.using() (by this script), on a page that does not have the category.

Currently, it does load the gadget, which is convenient for the case at hand. But one could argue that such mw.loader.load() should not load the gadget. Therefore, I would need to know if the current behavior is intentional and well-defined, or if it may change in the future (maybe even without notice).

Additionally, I have another, similar case: a gadget ("CommonEdit" here) that had (but may have again in the future) an "actions=edit" condition, but this gadget may also be loaded on a "view" page (by this script).

If you mean during preview, yes it will load if the last saved revision of the page had the category wikicode. What section you're editing is irrelevant.

Are you sure about this? It seems surprising that the last saved revision is taken into account. Especially considering the two next points (history and ajax previews) of your reply, where it's the opposite.

Another issue: I have a case with a gadget ("Diaporama" here) that is defined with a "categories" condition, but this gadget may also be loaded using mw.loader.load()/ mw.loader.using() (by this script), on a page that does not have the category.

Conditional loading rules don't apply while using mw.loader. This is well-defined behaviour. The RL module registry does not know about which user or page it is being invoked on. The only exception to this rule is the skins= condition, which does prevent the gadget from being registered in the unsupported skins.

If you mean during preview, yes it will load if the last saved revision of the page had the category wikicode. What section you're editing is irrelevant.

Are you sure about this? It seems surprising that the last saved revision is taken into account.

Ah, turns out I'm wrong. EditPage is smart enough to set up a fake revision to force the correct behaviour (the same trick which also causes things like self-transclusion to preview correctly).

Especially considering the two next points (history and ajax previews) of your reply, where it's the opposite.

I don't see how the other two points are connected.

Another issue: I have a case with a gadget ("Diaporama" here) that is defined with a "categories" condition, but this gadget may also be loaded using mw.loader.load()/ mw.loader.using() (by this script), on a page that does not have the category.

Every JavaScript should be implemented to be used stand-alone.

  • You should never rely that or how it is configured in Gadgets-definition.
  • At least for testing or developing you need to activate it manually.

That means:

  • Check, whether all modules are in place, or trigger loading by using() if not.
    • If mentioned in Gadgets-definition it is just ticked off, otherwise loading will start before proceeding.
  • Check the namespace if meaningful, abort if not appropriate.
  • Check user rights, if meaningful for offering page deletion.
  • Check mobile or skin condition.

If the gadget is loaded without explicit category, I guess nothing will happen.

  • It is just avoided to load and execute a gadget on every page, which is consuming resources.
  • Gadgets-definition will provide modules, perhaps in packages, in a more efficient way. If not, you will require them anyway.
  • If the conditions expressed by category are not met, no element with the expected attribute will be present in page, and the gadget cannot apply anything on those.