ResourceLoader 2018 Review
Open, HighPublic

Description

This task tracks actionables that came out of the ResourceLoader 2018 Review between @Catrope and myself.

Objective

Improve "time to interactive" and mediaWikiLoadEnd metric, by reducing the number of serial requests from 3 to 2.

Current state

(Sizes based on measures from en.wikipedia.org, desktop.)

Req 1: Startup module

Current size: 29 KB (network transfer) - 85 KB (uncompressed).

  • Defines mwNow, mwPerformance, isCompatible.
  • Fetches "Base modules request" (Req 2).
  • Once completed:
    • Registers sources, modules, and site config.
    • Process RLQ, which will initiate fetch for "Page modules request" (Req 3).
Req 2: Base modules

Current size: 49 KB (network transfer) - 178 KB (uncompressed).

  • "jquery" (41 KB).
  • "mediawiki" (8 KB).
  • "wikibits" (0.4 KB).

Problem

The startup module should be able to, on its own, start to handle mw.loader.load() calls from the page HTML, instead of having to make a second request first and waiting for that to come back.

Plan

In a nut shell:
  1. Make the "mediawiki.loader" client simple and small enough to simply include in the "startup" module.
  2. Load "jquery" and "wikibits" as a default-dependency implied for all regular modules.

The mediawiki.loader is already quite simple in terms of JavaScript logic. This is good because that means it won't be a problem to be compatible with the restrictions put on code in the startup module, namely that its syntax has to parse without errors in older browsers. Older browsers will never execute it, so usage of ES5 or ES6 methods is fine at run-time, but syntax-wise an older browser needs to be able to download and parse it, after which it essentially runs !isCompatible() and then returns gracefully.

The other requirement is for "mediawiki.loader" to not depend on jQuery. This is mostly done already, but there's a couple of call sites left that we have to migrate.

Once done, "jquery" can become a regular module that is loaded as dependency (in Req 3), thus eliminating Req 2.

For regular modules to be able to continue to assume jQuery, we need to re-introduce a concept of "default dependencies". And we should do it differently than what we did back in 2015 ("Default modules: mediawiki.user, mediawiki.util"), which (in retrospect) was implemented rather poorly (with a separate web request, using the synchronous top-queue).

High-level steps:

  1. Refactor mediawiki.js to not require jQuery. (Sub tasks below)
  2. Split off the "mediawiki.base" module from "mediawiki".
  3. Remove "mediawiki" module from the base module's request, and instead embed it directly the startup module response.
  4. Implement "default dependencies" concept.
  5. In Startup module, remove "Base modules request", instead listing those modules as "default dependencies". Then, in startup.js simply call startUp(); directly at the end of the closure.

Sub tasks for "Refactor mediawiki.js to not require jQuery"

startup.js:

wikibits.js:

mediawiki.js (to become "mediawiki.loader"):

  • mediawiki.errorLogger.js: Fine as-is (small, dependency-free, es3-compat).
  • mediawiki.requestIdleCallback.js: Fine as-is (small, dependency-free, es3-compat).
  • mw.Map: Keep (used for mw.config). But remove use of isPlainObject(). Decide to either use all own keys regardless of type, or just all keys plainly.
  • Message class: Not used by loader, and depends on mediawiki.html and mw.format Move to base.
  • mw.message: Not used by loader, move to base.
  • mw.msg: Not used by loader, move to base.
  • mw.messages: Keep (mw.Map-based registry for loaded messages).
  • mw.log: Keep (used by loader).
  • mw.format: Move to base. Has one call in loader, but trivial to avoid.
  • mediawiki.track: Move to base. (Depends heavily on special features of $.Callbacks) Maybe leave a stub in mw.loader for track(), given it's used for error reporting.
  • mw.loader:
    • addEmbeddedCSS: Fix to use plain array instead of $.Callback.
    • getMarker: We can remove this in favour of simply append to head if absent. – https://gerrit.wikimedia.org/r/429124
    • register: Remove or deprecate "dependencies as a function" feature. – https://gerrit.wikimedia.org/r/428406
    • addScript: Replace $.ajax with plain script.onload/onerror and callback. Promise only used in one place, with ".always()".
    • queueModuleScript: Replace overkill $.Deferred with simple callback parameter.
    • addLink: Use plain append/insert instead of jQuery.
    • checkCssHandles: Find a way to change mw.loader.using( ['site'] ) to a job so that it we won't need using() and $.Deferred.
    • doRequest: Inline simple version of $.param().
    • asyncEval: Uses $.globalEval() (easy to replace for our current browsers, per jQuery 3).
    • execute: Idem, uses $.globalEval().
    • execute: Fix $ reference to be window.$ to avoid ref-error when RL loads jQuery itself.
    • using: Depends on $.Deferred(), not used in loader, move to base. – (Blocked on: Internal callers)
  • mw.inspect(): Not used by loader. Move to base.
  • mw.html: Not used by loader, move to base.
  • mw.hook: Not needed, move to base.
  • $j alias: Not needed, move to base.
  • mwLoadEnd: Depends heavily on mw.loader.using(), and jQuery for when(), Deferred(), and more... Not needed here though, can be moved to the navtiming extension. – https://gerrit.wikimedia.org/r/#/c/434618/

Other stuff:

  • Simplify mediawiki.log.js

Startup response size breakdown

By @Catrope and @Krinkle (as of 7 May 2018)

Network transfer: 29 K
Uncompressed content-length: 85,299

Breakdown:

  • mwPerf + mwNow + isCompatible + nojs/NORLQ handling + startUp.addSource: 1,195 bytes (1.4%)
  • startUp.register: 67,999 bytes (80%)
    • jquery.*: 3013 bytes
    • mediawiki.*: 8532 bytes (core modules)
      • mediawiki.widgets.*: 1042 bytes
    • oojs*: 1073 bytes
    • skins.*: 344 bytes
    • schema.*: 2464 bytes
    • ext.*: 51,857 bytes (Extension modules)
      • CentralNotice: 1125 bytes
      • VE+Citoid: 2912 bytes
      • mobile.* + Minerva: 3588 bytes
      • Echo+Thanks: 862 bytes
      • RelatedArticles+RevisionSlider+TwoColConflict: 1786 bytes
      • ULS: 912 bytes
      • ContentTranslation (CX): 6573 bytes
      • Wikibase: 14,007 bytes
      • Gadgets: 4956 bytes
      • TMH/mwEmbed: 1426 bytes
  • startUp.config: 15,537 bytes (18%)
  • RLQ handling + addScript: 485 (0.6%)

Effort to reduce the size of the startup manifest tracked at T127328: Optimise critical rendering path.

Related Objects

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 434617 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/extensions/NavigationTiming@master] ext.navigationTiming: Move 'resourceloader.loadEnd' logic here

https://gerrit.wikimedia.org/r/434617

Change 434618 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] resourceloader: Remove mwLoadEnd code (moved to Navigation Timing)

https://gerrit.wikimedia.org/r/434618

Krinkle updated the task description. (Show Details)May 22 2018, 11:32 PM

Change 434178 merged by jenkins-bot:
[mediawiki/core@master] mediawiki.debug: Move internal footHovzer to the same module

https://gerrit.wikimedia.org/r/434178

Change 434364 merged by jenkins-bot:
[mediawiki/core@master] Remove deprecated 'es5-shim' module

https://gerrit.wikimedia.org/r/434364

Change 434617 merged by jenkins-bot:
[mediawiki/extensions/NavigationTiming@master] ext.navigationTiming: Move 'resourceloader.loadEnd' logic here

https://gerrit.wikimedia.org/r/434617

Change 434618 merged by jenkins-bot:
[mediawiki/core@master] resourceloader: Remove mwLoadEnd code (moved to Navigation Timing)

https://gerrit.wikimedia.org/r/434618

Change 434735 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] Remove the unused 'jquery.farbtastic' module

https://gerrit.wikimedia.org/r/434735

Change 434735 merged by jenkins-bot:
[mediawiki/core@master] Remove the unused 'jquery.farbtastic' module

https://gerrit.wikimedia.org/r/434735

Krinkle updated the task description. (Show Details)May 24 2018, 12:04 PM

Change 435291 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] Remove use of $.params in mediawiki.js

https://gerrit.wikimedia.org/r/435291

Krinkle updated the task description. (Show Details)May 26 2018, 10:03 AM
Krinkle updated the task description. (Show Details)May 30 2018, 2:28 PM

Change 435291 merged by jenkins-bot:
[mediawiki/core@master] resourceloader: Remove use of $.params in mediawiki.js

https://gerrit.wikimedia.org/r/435291

Change 436340 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] resourceloader: Create mediawiki.base.js and move mw.inspect to it

https://gerrit.wikimedia.org/r/436340

Krinkle updated the task description. (Show Details)May 30 2018, 6:53 PM

Change 436340 merged by jenkins-bot:
[mediawiki/core@master] resourceloader: Create mediawiki.base.js and move mw.inspect to it

https://gerrit.wikimedia.org/r/436340

Scripts dependent on resourceloader.loadEnd hook have broken. Was it planned? Was the hook removed?

Scripts dependent on resourceloader.loadEnd hook have broken. Was it planned? Was the hook removed?

This internal hook was not supported for use by gadgets. The public interface provides wikipage.content and $().

Looking at all wikis using mwgrep, I did find 7 uses, which I've now edited. Thanks for noticing! If there are uses elsewhere, please note that the page will not cause errors (no "Exception" thrown), no browser warnings, and other functionality remains unaffected.

This internal hook was not supported for use by gadgets. The public interface provides wikipage.content and $().

Well, it was present in the documentation (now it's not). As I remember, it was a good way to catch a very early moment which used to happened even before DOMContentLoaded has fired. Not sure if it was conceptually appropriate for our purposes though; we used it to coordinate loading order of many related scripts & gadgets. Anyway, we now don't need it where we did, since we now do most of the task with gadget dependencies.

This internal hook was not supported for use by gadgets. The public interface provides wikipage.content and $().

Well, it was present in the documentation (now it's not). As I remember, it was a good way to catch a very early moment which used to happened even before DOMContentLoaded has fired. Not sure if it was conceptually appropriate for our purposes though; we used it to coordinate loading order of many related scripts & gadgets. Anyway, we now don't need it where we did, since we now do most of the task with gadget dependencies.

Yes, it was automatically indexed by JSDuck, but it had no documentation explaining what it means or when it should be used.

As for what it actually was, this hook was fired when all ResourceLoader modules in the queue (including their scripts, styles, and interface messages) have finished downloading, parsing, and executed. This should always be after DOMContentLoaded, and sometimes even after the Window#load event. It was a low priority hook after which the queue is guaranteed to be empty.

According to data collected last month, for 70% of views it fired between DOMContentLoaded and Window#load, and for 30% of views after Window#load.

It's only purpose was to measure for statistically purposes how long it takes to load all modules. It was not a hook for doing in response when things are "ready".

It was not useful to use for other purposes because your code cannot control what it waits for. It will for any module on the current page. If the code needs a specific module, it should use mw.loader.using(). If the code depends on specific skin interface, it should use $() or for page content, wikipage.content.

I guess I'm incorrectly mixing DOMContentLoaded and moments of execution of its handlers here, here's one of my 2016 tests of in which order the hooks/events are fired on the action=edit page:

1480205746966 resourceloader.loadEnd
1480205747184 wikipage.content
1480205747356 wikipage.editform
1480205747566 ext.wikiEditor.toolbar
1480205747569 $
1480205748301 doneInitialSections

this hook was fired when all ResourceLoader modules in the queue (including their scripts, styles, and interface messages) have finished downloading, parsing, and executed

For some reason, in one of my recent tests I concluded that this hook doesn't guarantee that the modules have executed, not just loaded, but maybe I'm mistaken. But if you are right, why are you saying it's not relevant 'for doing in response when things are "ready"'? Our use case was to make sure all gadgets have loaded if they are switched on and run a script (dependent on those gadgets) afterwards. If we had just used mw.loader.using(), we would have loaded gadgets that the user didn't switch on.

Btw, I said

Anyway, we now don't need it where we did, since we now do most of the task with gadget dependencies.

I realized that it's not correct because, again, the gadgets are interdependent, and, in our "infrastructure", there is a core hidden gadget and a couple of shown gadgets dependent on it. But the hidden gadget has to make sure all the shown gadgets that the user has selected have been executed before running the main code. What techniques would you recommend for that arrangement? As for now, we only have $(), but as I learned after couple of years of script/gadget building it can be unreliable.

What techniques would you recommend for that arrangement?

If the intention is to schedule code after certain modules finish loading, without also creating a demand for a module that would otherwise not have been loaded, I recommend using mw.loader.getState() and mw.loader.using().

For example:

var pending = mw.loader.getModuleNames().filter(function (m) { return m.match(/^ext.gadget/) && mw.loader.getState(m) === 'loading'; });
mw.loader.using(pending).then(function () {
 // ..
});

Thanks, I guess that's it.

Krinkle updated the task description. (Show Details)Jun 5 2018, 1:45 AM

Change 437655 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] resourceloader: move Message methods from mediawiki.js to base module

https://gerrit.wikimedia.org/r/437655

Krinkle updated the task description. (Show Details)Jun 6 2018, 7:01 PM

Change 437914 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] [WIP] resourceloader: move track() from mediawiki.js to the base module

https://gerrit.wikimedia.org/r/437914

Change 437953 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] resourceloader: move hook() and html() from mediawiki.js to base module

https://gerrit.wikimedia.org/r/437953

Change 437655 merged by jenkins-bot:
[mediawiki/core@master] resourceloader: move Message methods from mediawiki.js to base module

https://gerrit.wikimedia.org/r/437655

Change 438129 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] resourceloader: qualify $ variable in script() call to handle the case of jQuery

https://gerrit.wikimedia.org/r/438129

Perhelion added a subscriber: Perhelion.EditedJun 9 2018, 2:10 PM

I don't see the real usefulness of removing modules. (It leads to exporting and separating the modules again for script makers.) Makes this not the term module pointless!?

I don't see the real usefulness of removing modules. (It leads to exporting and separating the modules again for script makers.) Makes this not the term module pointless!?

(For context for everyone else, this is about the UploadWizard change, Perhelion also commented at https://github.com/wikimedia/mediawiki-extensions-UploadWizard/commit/1b9ff2f90847167859dba7763430b938336c003b#commitcomment-29306894)

Some data about each registered module (including the name and dependencies) is loaded on every page, even pages that never use the module, as part of the "startup module". This has a small effect on the page load times, and if you have a lot of extensions all doing this, it becomes a noticeable problem. We were paying this cost for nothing because we always loaded all ~30 of these modules on Special:UploadWizard, and we loaded 0 of them anywhere else. Might as well use one module.

To my knowledge all these tiny modules were not really reusable anyway (they mostly depended on the rest of UW being loaded to do anything). But if you were using some of them in a gadget or something, we could probably bring them back.

I am guessing this is about the ghastly jquery.arrowSteps module. I recall now that there was some similar problem when we moved it to UploadWizard :/ Were you using any of the other modules?

Krinkle updated the task description. (Show Details)Jun 14 2018, 8:08 PM
Krinkle updated the task description. (Show Details)
Krinkle updated the task description. (Show Details)Jun 14 2018, 8:12 PM

Change 437914 merged by jenkins-bot:
[mediawiki/core@master] resourceloader: move track() from mediawiki.js to the base module

https://gerrit.wikimedia.org/r/437914

Krinkle updated the task description. (Show Details)Jun 15 2018, 3:33 PM

Change 438129 merged by jenkins-bot:
[mediawiki/core@master] resourceloader: qualify $ variable in script() call to handle the case of jQuery

https://gerrit.wikimedia.org/r/438129

Krinkle updated the task description. (Show Details)Jun 15 2018, 8:36 PM

Change 440605 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] [WIP] resourceloader: Embed 'mediawiki' directly in startup response

https://gerrit.wikimedia.org/r/440605

Krinkle updated the task description. (Show Details)Jun 15 2018, 8:56 PM
Krinkle updated the task description. (Show Details)

Change 437953 merged by jenkins-bot:
[mediawiki/core@master] resourceloader: move hook() and html() from mediawiki.js to base module

https://gerrit.wikimedia.org/r/437953

Change 440608 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] mediawiki.base: Move $j alias to mediawiki.base.js

https://gerrit.wikimedia.org/r/440608

Change 440608 merged by jenkins-bot:
[mediawiki/core@master] mediawiki.base: Move $j alias to mediawiki.base.js

https://gerrit.wikimedia.org/r/440608

Krinkle updated the task description. (Show Details)Tue, Jun 19, 3:24 PM

Change 441217 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] resourceloader: remove internal using() dependencies in mediawiki.js

https://gerrit.wikimedia.org/r/441217

Change 441218 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] resourceloader: move using() from mediawiki.js to the base module

https://gerrit.wikimedia.org/r/441218

Od1n added a subscriber: Od1n.Fri, Jun 22, 1:32 AM

I think the legacy "mediawiki.api.*" modules should be removed later than in MediaWiki 1.33.

The current LTS, MediaWiki 1.31, only supports the legacy modules, so I think we should support both modules systems (i.e. merged modules, and submodules with deprecation warnings) until the next LTS, on which we could make the removal of legacy submodules.

I think the legacy "mediawiki.api.*" modules should be removed later than in MediaWiki 1.33.

The current LTS, MediaWiki 1.31, only supports the legacy modules, so I think we should support both modules systems (i.e. merged modules, and submodules with deprecation warnings) until the next LTS, on which we could make the removal of legacy submodules.

That's not the deprecation policy. If you think it should be, you should suggest it at https://www.mediawiki.org/wiki/Talk:Deprecation_policy.

Od1n added a comment.Fri, Jun 22, 1:41 AM

Thanks :)

I just found it was cumbersome to manage different versions of scripts, depending on MediaWiki version, swimming on short intervals like 6 months.
Moving to forward-compatbility implying to break backward-compatibility even for recent and supported MediaWiki versions.

(TBH, I'm quite tired of the incessant and short-notice javascript deprecations. Case in point, wikis are full of broken scripts, even amongst the most used ones, only because of the deprecations...)

(TBH, I'm quite tired of the incessant and short-notice javascript deprecations. Case in point, wikis are full of broken scripts, even amongst the most used ones, only because of the deprecations...)

Can you please provide one or two examples of short-notice deprecations and how which wiki was full of broken scripts due to this? I can count on one hand, the number of deprecations and removals we have performed in JavaScript in 10 years. They are far from short-notice.

Krinkle updated the task description. (Show Details)Fri, Jun 22, 6:14 PM

Change 441586 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] mediawiki.hook: Move mw.hook tests to new mediawiki.base.test.js file

https://gerrit.wikimedia.org/r/441586

Od1n added a comment.EditedSat, Jun 23, 12:45 AM

Not strictly deprecation/removal, but off the very top of my head:

  • Requiring to add the mediawiki.util dependency. This made many scripts significantly more cumbersome, not to mention the insane amount of work to update the scripts. As more, this module is nevertheless always loaded at some point in practice, and it is quite short (also, you could compare this with the removal of the "mediawiki.api.*" submodules, where "a small size increase is acceptable in order to simplify").
  • Implementing jQuery UI, then deprecating it and switching to that "OOUI-JS" thing. Not to mention I hate it (complicated code, lacks flexibility, very slow), some scripts still use jQuery UI and don't switch that OO-JSUI because the code is complicated and often doesn't let do some required things.
  • Many markup changes introduced by that OO-UIJS, that broke many scripts.
  • A lot of ranting because of that OOuiJS (just a few examples: 144171441, 144171817, 144172971)
  • Another one: 141529955 (in edit summary: "class has been removed. script already broken three months after. f***.")
  • Bonus: giving the precious $ global name to jQuery. jQuery is slowly dying, and the day it will be to be removed, we're fucked. Already as of now, all documentation should make use the jQuery global name, using an IIEE to get the $ name.

TL;DR: Off the top of my head: explicit mediawiki.util dependency, everything involving OOUIjs (mainly the markup changes). I miss probably some, I just can't think of them right now.

This long message is going slightly off-topic, and I don't really have time to further discuss about it. Just consider it as some feedback, which I hope might be useful in some way.

Od1n added a comment.Sat, Jun 23, 3:43 AM

I'll just also mention toolbar customization, which has been modified several times, and has become a real pain for users.

See this past discussion, and the addCustomButton function I had to add in the Common.js so that it becomes back sanely usable.

@Od1n Thanks for taking the time to document these points. I understand your frustration. Having said that, I will provide some details behind these changes that I think are important.

Gadgets provide (virally elected) users the ability to develop and deploy software with no requirements, no technical limitations, and no review from professional developers. This is a big risk in terms of performance, security, privacy and usability. Yet, we grant users this power. But power comes with responsibility. And developing software, regardless of whether as a hobby or professional, comes with the responsibility to maintain it. Volunteer and staff developers that contribute to MediaWiki software, perform maintenance. This ensures our wikis remain secure, user-friendly, compatible with constantly changing browsers and devices, and (hopefully) fast for everyone.

Sometimes this maintenance must change code in a way that is not backward compatible, due to mutually exclusive requirements. When requirements are not mutually exclusive, we follow a deprecation phase in one of two ways:

  • Build a new bridge that allows references to the old code to keep working by automatically mapping to newer code.
  • Or; by keeping a copy of the old code unchanged.

However, sometimes the requirements are mutually exclusive in a way that the old code cannot work anymore. When this happens, the deprecation happens before adding the new code, instead of after adding the new code. We do this by building a new bridge to the old code in a way that is prepared for the new requirement. Then code is updated to this new bridge, and once we can remove the old, we can add the new code and point the bridge to that. This means during the transitional period code must find a common ground that requires neither the old nor the new behaviour specifically.

All of this costs time, money, and (sometimes) affects site performance for all other users. This is considered worth it, because it is important to have a transitional period to allow existing code to be migrated. This applies to both core and extensions (maintained by MediaWiki volunteer and staff developers) as well as to gadgets and user scripts (maintained on-wiki).

However, these bridges cannot be maintained forever. Doing so would cause the performance to get worse every time, would cause to cost increasingly more time and money, which is not sustainable. Even if it were sustainable, it would be insecure, because old code often relies on old libraries from other organisations which will not find or fix security flaws in their older versions and web pages would need to load multiple versions of logically equivalent code, which makes performance even worse.

With that, I'll respond to three of your examples.

  1. Requiring to add the mediawiki.util dependency made many scripts significantly more cumbersome, not to mention the insane amount of work to update the scripts. [..] this module is nevertheless always loaded at some point in practice, and it is quite short.

It was never supported for users to write code using mw.util without a dependency added. Using modules without depending on them was never announced or allowed in documentation. Writing code in this way is a mistake. Fortunately, fixing it needs only one line of code to change.

Despite this mistake, such code worked by accident due to a race condition whereby your code coincidentally runs after unrelated other code that in turn did specify a dependency. This means broken code "works" as long as at least one other gadget that needs the same module, is not broken and also happens to load before yours loads.

In other words, your module without a dependency was working because someone else's module did have the dependency.

This is true for all modules, but you are right that mediawiki.util is very commonly used. This is why, despite it not being supported by the documentation, people that tried this did not always get an error. But... as expected, it did not work always, and sometimes it did cause an error.

We never added a requirement to add it. In fact, we did the opposite. Following reports at T35746 in 2012 from some people getting an error, I added a special feature for "mediawiki.util" called $wgPreloadJavaScriptMwUtil which, when turned on, will load it always, before anything else. This comes at a performance cost, because it delays everything on the page, including parts that do not need it. It also makes tampers with automated load order and prevented us from making other improvements, which had to wait for many years.

After three years, $wgPreloadJavaScriptMwUtil was removed in 2015. But, alas, mediawiki.util was still loading by default for gadgets because we also have the feature "$wgIncludeLegacyJavaScript" for wikibits.js, which was still turned on, and wikibits depended on mediawiki.util. The legacy wikibits.js migration that started in 2011 and was still not completed after four years in 2015.

Before I continue, let me address the second part of your point:

  1. [..] this module is [..] quite short.

The mediawiki.util.js file is 5.5 KB (minified). One such thing is fine, but many of them add up. Also, the cost of mediawiki.util is more than this one file.

For four years (2012 – 2016) mediawiki.util was manually loaded by default specifically to help gadgets. During these years, we reached out via Wikitech-l, Wikitech-ambassadors, Tech-News, and console deprecations. In addition to reaching out via these ways to users that maintain gadgets, @Krenair, @matmarex, @Legoktm and myself also spent hundreds of volunteer hours ourselves to look on all 900 wikis and proactively help address this migration in most gadgets that needed it.

In 2016, I decided to allow wikibits to continue for one more year. But optimise it so that it does not need mediawiki.util. This improved performance significantly because it allowed us to remove one layer of indirection. This was a relative improvement, but really just restored how things were in 2012.

I hope that later this year (in 2018), we will finally be able to remove one more layer of indirection (current task is about that), which would make number of network steps finally better than before 2012.

  1. Implementing jQuery UI, then deprecating it. [..] complicated OOUI code, lacks flexibility, very slow [..]

jQuery UI was first introduced to MediaWiki in 2009, with provision through ResourceLoader in 2011. In 2014 (four years later) we upgraded from jQuery UI 1.8 to jQuery UI 1.9, which included a compatibility layer for jQuery UI 1.8 (T49076).

Several gadget authors in the community, staff engineers, and other developers have asked for an upgrade to jQuery UI 1.10, or jQuery UI 1.11. However, we have not done so despite the bugs that exist in jQuery UI 1.9, and despite the improvements the newer versions have. This, because it means ending the compatibility with jQuery UI 1.8. (Each jQuery UI version has compatibility for the previous version.)

In August 2016, after actively using jQuery UI for seven years, we marked it as deprecated in favour of OOUI. Today, two years since the deprecation started, jQuery UI 1.8 code still works in production. We have still not removed it. You shouldn't, but if you don't have time yet or still have not found an adequate replacement, it can continue to work with jQuery UI at this time.

If you have specific issues with OOUI, please create a task with a specific need and expectation under OOUI so that the team looking after it can help you.

jQuery UI 1.9 is now so old, the jQuery Team no longer supports our version. This means for the past four years, I have had the additional responsibility (distracting me from other duties) to manually customise our old version of jQuery UI to be compatible with current browsers and current jQuery core 3.0 version.

  1. giving the precious $ global name to jQuery. jQuery is slowly dying, and the day it will be to be removed, we're fucked. Already as of now, all documentation should make use the jQuery global name, using an IIEE to get the $ name.

This is incorrect.

jQuery's official alias is $ and is part of jQuery. This is not something we gave it. In either case, we support the alias, and neither name is deprecated. Both are 100% supported, current and stable.

There is no need for an IIFE for that. As long as jQuery exists on a Wikimedia page, $ will exist and point to jQuery. If for some bizarre reason we find a really good reason to change $ to something else, there will be deprecation warnings, announcements, and many years of transition before anything changes. This is not a concern.

Also, jQuery is not dying. Slow, or otherwise.

I'll just also mention toolbar customization, which has been modified several times, and has become a real pain for users.

See this past discussion, and the addCustomButton function I had to add in the Common.js so that it becomes back sanely usable.

The Classic toolbar API was modified two or three times over the course of a decade. That is very low frequency for software, even for Wikipedia where things usually change slowly.

The supported interface actually only changed once (from mwCustomEditButtons to mw.toolbar, with a three-year deprecation phase). The other two changes you saw were internal, but affected some users that invented their own unofficial ways which then stopped working. If there is no manual explaining how to do something, ask a developer. If you decide to do yourself without documentation for your purpose, then be prepared that it can stop working without announcement or deprecation. (More likely, it changes with an announcement that sounds unrelated.)

If two changes per year is a problem, I recommend thinking about whether you really want to contribute to program code. Also, if something unofficial does stop working, you can still ask and someone will still help (I recommend Phabricator or Wikitech-l mailing list). That is considered a success, not a pain.

Od1n added a comment.EditedSat, Jun 23, 4:49 AM

Wow, that's quite an impressive reply! Thank you. Pinned.

I do understand the need to move forward and break legacy drags.
Problem is, there are thousands of user scripts, stacked for over a decade. Some time ago, I spend a lot of time and efforts on fixing such scripts, and there is a lot remaining to do. If I had not fixed them, probably no one else would have done it. And as I'm not planning to, and can't, spend more time on it, the rest will probably just stay indefinitely broken...

About the now widespread trend of fast updates, here's some of my PHP development experience: some PHP frameworks are great, but they now require to update the application every 6 f— months. For just one project, it's already annoying, but acceptable. But if you have dozens of projects, constantly updating them is just insane and beyond feasibility. So you have to either let them rot, or change paradigm (e.g. centralize apps, or change framework).

Just my thoughts. Your message above is a much more elaborate and accurate description of the situation.

@Od1n Thanks, I now understand where the frustration comes from, and I think we agree. I've had the same experience with scripts. Probably many of them were not created by you, but they seem broken, and nobody else would fix it?

I believe we (as community members) need to discuss and decide how we want to solve this problem. I think it is okay for global interface editors, sysops, and stewards to maintain common gadgets on many wikis, and to maintain a few local scripts on small number of wikis they actively care for. But I do not think we can maintain all local scripts on all wikis, right? The math is simple. If a wiki wants custom logic, somebody needs to create it, and it needs to be actively looked after by someone.

Perhaps we (community) can organise some way for local communities to help find volunteers that know JavaScript and speak their language.

Change 441217 merged by jenkins-bot:
[mediawiki/core@master] resourceloader: remove internal use of using() in mediawiki.js

https://gerrit.wikimedia.org/r/441217

Change 441218 merged by jenkins-bot:
[mediawiki/core@master] resourceloader: move using() from mediawiki.js to the base module

https://gerrit.wikimedia.org/r/441218

Od1n removed a subscriber: Od1n.Sun, Jun 24, 12:33 AM
Krinkle updated the task description. (Show Details)Sun, Jun 24, 2:35 AM

Change 441220 had a related patch set uploaded (by Aaron Schulz; owner: Aaron Schulz):
[mediawiki/core@master] resourceloader: spin base module code out as a proper module

https://gerrit.wikimedia.org/r/441220

Change 441586 merged by jenkins-bot:
[mediawiki/core@master] mediawiki.hook: Move mw.hook tests to new mediawiki.base.test.js file

https://gerrit.wikimedia.org/r/441586

Krinkle updated the task description. (Show Details)Mon, Jun 25, 2:59 PM

Change 441220 merged by jenkins-bot:
[mediawiki/core@master] resourceloader: spin base module code out as a proper module

https://gerrit.wikimedia.org/r/441220

Krinkle updated the task description. (Show Details)Mon, Jun 25, 5:46 PM

Change 440605 merged by Krinkle:
[mediawiki/core@master] resourceloader: Embed 'mediawiki' directly in startup response

https://gerrit.wikimedia.org/r/440605

Krinkle updated the task description. (Show Details)Thu, Jul 12, 7:51 PM

Change 445462 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] [WIP] resourceloader: Combine base modules and page modules requests

https://gerrit.wikimedia.org/r/445462