Page MenuHomePhabricator

Proposal: support for ES Modules via script type="module" tags
Open, Needs TriagePublic

Description

ResourceLoader now supports scripts written in modern JS format via the new es6 option in module definitions. In the same vein, I'd like to propose doing the same thing for <script type="module">.

Support for ES Modules is pretty widespread at this point. In terms of our own Grade A browsers, the only blockers would be Safari 9 and legacy Android Browser (4.x series). Safari 10 can be polyfilled (but does not support the <nomodule> attribute). We could manage the inconsistent support landscape with a config flag similar to es6: true (module: true perhaps).

Shipping module scripts has several immediate benefits:

  • each module has its own isolated scope, so no more need to write IIFEs,
  • "use strict" is always in effect
  • module scripts are always deferred (same as the defer attribute), meaning that they are not blocking, load in parallel, and won't execute until the document is ready
  • module scripts will always execute in the order they are placed in the document
  • developers will benefit from better IDE tooling and will be able to write code in a more standardized way

Actual implementation of this change in ResourceLoader seems like it would be pretty straightforward (the RL client already appends a script tag for each module to document.head; telling it to add script.type = "module" for certain modules would not be too hard). But the bigger questions will of course concern the impact of such a feature on the larger MediaWiki ecosystem.

Thoughts?

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

developers will benefit from better IDE tooling and will be able to write code in a more standardized way

What do you mean by this exactly? If you mean import and export, then I think there would be additional challenges in getting those to work/coexist with ResourceLoader. Are there other language features that are only available when type="module" is used?

each module has its own isolated scope, so no more need to write IIFEs,

We would still need to wrap modules in anonymous functions in order to provide the require function and the module.exports object (unless we find other ways to do that).

module scripts will always execute in the order they are placed in the document

This could be a way to manage execution order (rather than storing anonymous functions and executing them when we know their dependencies have been executed), except that error handling could be different. Currently, we don't even attempt to execute a module if the execution of one of its dependencies threw an error. If ordered module scripts don't work analogously, that would change the behavior slightly (but probably only slightly, since modules often require() their dependencies, and we could throw an exception when something attempts to require() a module that errored out; or we could inject a preamble that verifies the dependencies' status).

developers will benefit from better IDE tooling and will be able to write code in a more standardized way

What do you mean by this exactly? If you mean import and export, then I think there would be additional challenges in getting those to work/coexist with ResourceLoader. Are there other language features that are only available when type="module" is used?

This is admittedly a bit vague, but I'm imagining an ideal future where a developer could write

import { Api, Title } from '../resources/mediawiki.js

inside their own source code, and get type hinting, "jump to definition" functionality, etc. As opposed to what we do now, where you have to rely on the implicit existence of a global mw object (which must also be mocked out in testing if you're running unit tests in node). Similar to the experience of writing PHP code in our projects, basically.

Change 684663 had a related patch set uploaded (by Eric Gardner; author: Eric Gardner):

[mediawiki/core@master] [WIP] Add support for <script type="module"> tags

https://gerrit.wikimedia.org/r/684663

This is a quick somewhat rejective reply, to avoid stalling later and start iterating sooner. Disclaimer: I mean only what I say, nothing further implied; any apparently implied opinions are either unintentionally expressed or weakly held. Please feel free to challenge anything/everything.

Overall I'm very much in favour of aligning us with ES modules and levering native browser features where possible.

  • each module has its own isolated scope, so no more need to write IIFEs,

This is already solved by package files which gives each file its own scope. If some legacy code was ported to package files but left closures behind, then can remove those right now.

  • module scripts are always deferred (same as the defer attribute), meaning that they are not blocking, load in parallel, and won't execute until the document is ready

As of 2015, all MediaWiki JavaScript is delivered asynchronously and considered logically optional.

There are some small differences between script async and script deferred (similar to script type=module). Mainly relating to when to interrupt the rendering thread, which in turn impact whether or not we make optimal use of otherwise idle network bandwidth capacity, and whether or not we artificially delay when interaction capabilities are bound. Changing this is very much up for debate, but imho better done separatey and motivated by different reasons. Those reasons could in turn inform us that we don't want to deliver code using the module/deferred mode of delivery if this is undesirable for performance. I believe we can keep this as an invisible implementation detail that does not affect any part of how code looks in version control or how we work during development.

  • module scripts will always execute in the order they are placed in the document

I'm not sure if this will be significant to us given bundling and caching. I don't expect us to have more than one script tag specified in the HTML (only the startup module), and anything beyond that is inserted at run-time subject to the same racces, regardless of any type=module attribute. So it would not actually be order that is meaningful or predictable. It would be randomly decided at runtime and then "ordered" from then onwards. It seems like all that does is take away the the browsers ability to re-order execution based on available capacity and downloads finishing, which seems worse for performance, not better. Dependency relationships are already upheld of course. But it's for the stuff where it doesn't matter where we'd lose the "whichever comes first" benefit.

I haven't looked deeply into this, but based on what I know now, there would not be a (different) benefit we would gain in return for that compromise. If you know of one, very much open to hearing that.

  • developers will benefit from better IDE tooling and will be able to write code in a more standardized way

I assume IDEs support require() about as well as import. Is that not the case? (I think you meant this point in relation to globals, see below.)

[…] As opposed to what we do now, where you have to rely on the implicit existence of a global mw object […]

I agree with this, and this is a point in favour of not using global variables and using a module system for linking between files.

For example, mediawiki.util already supports being require'ed in addition to exporting a global property. Is it easier to make an IDE resolve module names with import keywords, compared to making it resolve module names in require() calls?

@Krinkle fair enough – I agree that ResourceLoader currently gives us most of the benefits that <script type="module"> would already, with greater backwards-compatibility. However, I do think it's good to keep an eye on the evolving state of the web here. There may be a time when we want to do less of this work ourselves and rely more on built-in browser functionality.

I believe that there are benefits of encouraging greater use of import statements (and potentially module scripts as well) during development – even if we continue doing things in a more traditional way for deployed production code.

The traditional way we write code looks like this:

// OOUI is specified as a dependency for this module, so it is just assumed to exist as an implicit global.
// No editor hints, code completion suggestions, etc in the IDE.
// Linting and headless unit tests require extra configuration.
// The entire OOUI library – or at least a chunk of it – must be loaded at runtime
var buttonWidget = new OO.ui.ButtonWidget( {
    label: 'ButtonLabel'
} );

If we started using a bundler like Rollup or Vite (per T279108), we could get away from the implicit global and write something like this instead:

// Pull in OOUI from NPM, which means we can use it in unit tests more easily
// The name of the package in NPM is different from the name of the RL module so some workaround is needed
// to get them to line up
var OOUI = require( 'oojs-ui' );

// Still no editor hints or code completion in IDEs however
// Larger OOUI library still gets loaded here
var buttonWidget = new OOUI.ButtonWidget( {
    label: 'ButtonLabel'
} );

However, if we used a tool like Rollup or Vite but also switched over to import syntax in our source code, we could start to see some additional benefits:

// Import only exactly what is needed; works in the browser and in Node
// If using a bundler, it's possible to *only* include MyButton in the bundle and omit the rest of the library
import { MyButton } from 'my-new-library';

// If 'my-new-library' is properly set up, an editor like VSCode will tell me that I can provide an option 
// called "label", that it excepts a string, whether or not it is optional, while I type
let button = new MyButton( {
    label: 'ButtonLabel'
} )

In the final example, we could even go one step further and dispense with bundling entirely if desired – this could also remove the need to deal with a package manager like NPM during development. If all the files that make up my-new-library are vendored in MW core and written using import/export statements, and if we delivered the code that consumed this library using <script type="module">, then the user's browser could fetch my-new-library/MyButton.js without the need for any additional tooling.

The last step obviously represents a big break with the way we do things now, but it's possible we'd want to do things this way at some point in the future. Or perhaps we'd want to use module scripts this way for development only, similar to how Vite works.

Using import statements during development and then creating bundles that are delivered as normal <script> tags (converted to CommonJS/IIFE/UMD format) could be a good balance.

On a related note, it would be great if there was a way to import or require various MW utils (Title or Api for example) in development without needing the larger MW environment (importing the code either from the filesystem or from NPM); that would make it easier to start working this way within our projects. Such dependencies could be "externalized" if a bundler was being used so that they don't end up being inlined in the deployed bundle during production (where RL can continue to provide the code as it does now).

On a related note, it would be great if there was a way to import or require various MW utils (Title or Api for example) in development without needing the larger MW environment

Big +1 to everything you've said here. Particularly the VSCode benefits which are huge for developer productivity.

Note these goals resonate me and were chiefly the reason I raised T212521: RFC: Reconsider how we run QUnit unit tests

FWIW Note I think we should be working towards publishing all mediawiki core resource loader modules as npm libraries. One way to work towards this is using packageFiles and package.json files inside individual ResourceLoader folders like in the proof of concept attached to T212521