Page MenuHomePhabricator

Add native support for ES2016-ES2020 or higher versions
Open, LowPublic

Description

Now that we have done T272104: Allow modules to opt-in to ES6 syntax support, we should consider doing the same thing for ES2016, ES2017, or whatever version makes sense.

We will have to consider the benefits of new language features in a given ES version versus the costs of dropping browsers that don't support it, and the complexity of feature detection.

Notable syntax features in later versions:

  • ES2016: the ** operator.
  • ES2017:
    • async function declarations.
    • await statements.
    • async function expressions.
    • async function methods.
  • ES2018: async iteration (for await), spread (...) in object literals, regex features (named capture groups, lookbehind, /s flag, \p{...})
  • ES2019: optional catch binding.
  • ES2020: ?? nullish and ?. operators, dynamic import, import.meta, export * as Foo from 'bar', BigInt.

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Catrope updated the task description. (Show Details)

I haven't yet looked at which browsers support these features and how well-used they are. That information should be added to this task next. I don't have time to do that right now; anyone else should feel free to pick that up.

Krinkle moved this task from Inbox to Backlog on the MediaWiki-ResourceLoader board.
Krinkle subscribed.

Lowering priority for now this is not purely a backend enhancement, it's a product decision impacting browser support. That decision will inform when resourcing for this is needed.

Note that this task is purely about syntax sugar, not methods and APIs. APIs for Array and Object methods etc can and should be skip-loaded polyfilled, the same as with any other Web APIs (e.g. those not part of the ES spec).

You can find browser versions and features in Kangax tables (at least a few versions back):
http://kangax.github.io/compat-table/es2016plus/

Note that Chrome 49 (2016-03-02) was the last version for Windows XP (I still see XP in my analytics, but YMMV). Anyway if you wish to support XP ES2016 would be the last to consider I guess. If XP is not a problem, than browsers are evergreen now, so anything really within 2-3 years would be OK.

Chrome 52
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Exponentiation#browser_compatibility
Chrome 47
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/includes#browser_compatibility

Change 891938 had a related patch set uploaded (by SD0001; author: SD0001):

[mediawiki/libs/Minify@master] Support ES2016 syntax in JS minifier

https://gerrit.wikimedia.org/r/891938

Change 891938 merged by jenkins-bot:

[mediawiki/libs/Minify@master] JavaScriptMinifier: Add ES2016 syntax support

https://gerrit.wikimedia.org/r/891938

Krinkle renamed this task from Explore native support for ES2016 (ES7) or higher versions to Add native support for ES2016-ES2020 or higher versions.Mar 7 2023, 12:18 AM

Happy to help with the mechanics of getting a new release of Minify out and in prod.

Change 990374 had a related patch set uploaded (by SD0001; author: SD0001):

[mediawiki/libs/Minify@master] Add a few tests for ES2017 features

https://gerrit.wikimedia.org/r/990374

  • async function methods.

I assume this is referring to async methods in classes and/or object literals. But it seems to work? I added a few tests. People have also started using it in production wikis. @Hokwelum @Krinkle I see the commit message says they are not yet handled. Is there a reproducible edge case where it fails?

Another ES2017 feature not mentioned in the description is the support for trailing commas in function declarations and calls. In this case also, the minifier does give valid output, although it misses to optimize away the trailing comma.

Change 990374 merged by jenkins-bot:

[mediawiki/libs/Minify@master] Add a few tests for ES2017 features

https://gerrit.wikimedia.org/r/990374

  • async function methods.

I assume this is referring to async methods in classes and/or object literals. But it seems to work? […] Is there a reproducible edge case where it fails?

These are not yet supported. They work by accident on most inputs. This is true for all syntaxes including for much ES6 inputs in the years before we supported it officially. The parser is tolerant and recovers on its own. The debug mode output in the JavaScriptMinifier class will show that it does not recognise these tokens correctly.

Ultimately all the minifier has to decide is 1) whether whitespace can be removed and 2) if not, whether to condense a space or a new line, and 3) whether we can insert a periodic line break at this point.

As such, even very advanced and unsupported syntaxes will often minify correctly today. More complex examples where expressions are nested and e.g. newline-sensitive tokens are used (such as return or await), will consitently result in output that contains a syntax error and/or has a different meaning.

This doesn't work today because MediaWiki core requires ES6 and the gadget syntax validator allows up to ES7.

It can of course be saved (pending T76204), and loaded outside ResourceLoader's checks via action=raw or via the temporary requiresES6 option in gadgets. However, misuse of requiresES6 to store ES8 code is not supported and will stop working after we remove that flag (announced in Tech News at T75714#9442330). The flag was added because we did not yet have a strategy for updating syntax validation. The new solution allows us to trivially raise it in a one-line change any time that MediaWiki core's browser requirements change. There are no plans currently to raise browser requirements to ES8/ES2017 or ES2020 etc. Not likely until browser engines, browser vendors/apps, audience usage thereof catch up.

Note that this task is only about preparing the minifier.

Please, please keep the requiresES6 flag at least until ES8 can be used in scripts. On frwiki, we have scripts that use async/await, and this language feature makes the codes incredibly simpler. We even had codes with the ?. operator, but I rewrote them to avoid using this operator (for now). But the async/await is a must have.

Alternatively, another temporary solution could be to add an esversion=integer option, which thanks to its more precise control, would replace requiresES6 and its pitfalls. I know the plan is to make all scripts (core, gadget, site, user) use the same ES version, just suggesting a better temporary solution.

Well... we did tell people not to do that.. its not called requiresES6 for nothing.

There are no plans currently to raise browser requirements to ES8/ES2017 or ES2020 etc. Not likely until browser engines, browser vendors/apps, audience usage thereof catch up.

I fought with myself a bit about introducing async/await, but I wouldn't be able to keep the process sane and readable without async/await here:
https://github.com/Eccenux/wiki-DYKCzyWiesz/blob/main/src/DykProcess.js

So I'm definitely for keeping requiresES6. I did have to use that option to make a new Did-You-Know tool work. So I can also confirm that async/await didn't work without requiresES6.

Fun fact. I just learned that chrome status page for async/await is so old that most links on that page are dead already (including the MS demo page and TC39's page). Also Github shows 10 million matched files for await which probably many of them are not for await/async, still that seems like a lot. So I think it is fair to say the feature is very much established. Also interesting fact both jQuery.ajax and mw.Api can be used with await. And the code is SO MUCH better with await.

So please do consider upgrading to next ES.

@Nux Standardisation is not in question. Unlike new Web APIs (such as fetch), which can be polyfilled, or used via a conditional "if" statement, new syntax, if allowed even once, immediately breaks all code in many browsers that we still support. You can complain to upstream TC39 for having introducing features in ES2017 in a way that is backwards-incompatible. When MediaWiki raises its minimum requirement to ES8/ES2017, we will on the same day, raise the ESLint and site script validator to also allow this syntax.

The validator already supports all ES versions upto and including ES2024. The raising of our configuration is a product decision about audience and browser support. It is not a technical decision.

Meanwhile, please keep this task on-topic. This task is a technical task, about the minifier, not the validator.

So please do consider upgrading to next ES.

There is nothing for us to upgrade. We intentionally restrict the allowed syntax based on what the browsers used by Wikipedia readers, support. See T178356#8740573 for a peak behind the curtain, in which I lay out the reality of browser support.

The marketing of "evergreen browsers" is mostly a lie spread by privileged developers in the west, given ever steeper and shorter lifecycles for OS/browser support, and also a growing landscape of alternative browsers on Android that embed widely outdated copied of browser engines (e.g. in 2023, the latest version of UCWeb browser on Android, seemingly embedded a version of Chrome from several years older; likewise, iOS and Android devices are often unable to upgrade to a newer browser due to discontinued vendor support).

In any event, the topic of browser support raising is a separate one. Once usage is at an acceptable level, I expect WMF will raise Grade A requirements again, as we always have.

@Nux Standardisation is not in question. Unlike new Web APIs (such as fetch), which can be polyfilled, or used via a conditional "if" statement, new syntax, if allowed even once, immediately breaks all code in many browsers that we still support. You can complain to upstream TC39 for having introducing features in ES2017 in a way that is backwards-incompatible. When MediaWiki raises its minimum requirement to ES8/ES2017, we will on the same day, raise the ESLint and site script validator to also allow this syntax.

I was on the TC39 lists. Changes discussed were far more breaking than they are, even for ES6 (well for Harmony back then). So I'm not complaining. Also, it was then 10 years ago. Back then, I didn't even like building JS with JS and also had a PHP compiler and was working on a JQuery Mobile framework... That was a long, long time ago, and I've moved on. Wikipedia should move on too :)

Meanwhile, please keep this task on-topic. This task is a technical task, about the minifier, not the validator.

I am talking about the minifier. Or at least I think I am. The minifier was breaking on this version of the DYK gadget:
https://pl.wikipedia.org/w/index.php?title=MediaWiki%3AGadget-CzyWiesz.js&diff=72699054&oldid=72675626

There was some exception in JS console. It started to work after adding requiresES6 in this change:
https://pl.wikipedia.org/w/index.php?title=MediaWiki%3AGadgets-definition&diff=72699123&oldid=72407946

I'm using eslint with "es2017": true, so I'm fairly sure the code is compatible with ES2017, so the minifier is not.

The marketing of "evergreen browsers" is mostly a lie spread by privileged developers in the west, given ever steeper and shorter lifecycles for OS/browser support...

I'm quite aware. I work for a company doing software for Polish libraries. Some of them still use Windows XP. As it so happens, though:

In 2017, the last version of the Firefox browser available for Windows XP was released (Firefox 52 ESR)[3][27].

And that is the version that supports async :-). So you know... https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function#browser_compatibility

2023, the latest version of UCWeb browser on Android, seemingly embedded a version of Chrome from several years older; likewise, iOS and Android devices are often unable to upgrade to a newer browser due to discontinued vendor support

Yes, browsers are discontinued, and being evergreen is a lie, you are right. But they are not that old. Even KitKat had updates beyond what Windows XP had.
https://groups.google.com/a/chromium.org/g/chromium-dev/c/ypAS49lvN1M/m/LC0OFek5EAAJ?pli=1

So you know. It's not super, but it is not as bad as you might think :) And the point is to make the gadgets work. Not to raise compatibility for everything. Just for gadgets that people choose to use (or not). So as said - please keep the option requiresES6. Name it requiresESnext if you must, but please keep it.

Let's continue the discussion on Gadgets in T357197, since it is indeed off-topic (or only tangentially related) to this task.

Change #1048590 had a related patch set uploaded (by Krinkle; author: Krinkle):

[mediawiki/libs/Minify@master] JavaScriptMinifier: Add support for `**=` operator (ECMAScript 2016)

https://gerrit.wikimedia.org/r/1048590

Change #1048590 merged by jenkins-bot:

[mediawiki/libs/Minify@master] JavaScriptMinifier: Add support for `**=` operator (ECMAScript 2016)

https://gerrit.wikimedia.org/r/1048590