Page MenuHomePhabricator

Use CompressionStream API where available (instead of Pako polyfill)
Open, MediumPublic

Description

Standardisation effort has recently begun on exposing compression mechanisms, which browser have held natively for decades, to JavaScript execution.

If this comes to fruition, it means VisualEditor won't have to download the easy-deflate.deflate pako's deflate module anymore before it can submit edits. This module currently has an upfront cost of 10 KB 8800 bytes (in practice 11.5K, see note) in bandwidth, 32 27 (pre-minified) KB in source code that requires parsing and execution. (based on easy-deflate.core + easy-deflate.deflate pako).

NOTE: "8800 bytes" was said in T236210, but that's for the pre-minified version as provided by the pako project. It appears MediaWiki includes the full version and relies solely on ResourceLoader to minify it. Search the downloaded code for "arraySet", the function arguments would be shortened in the pre-minified version. The ResourceLoader-minified module is ~11.5K after gzip.

In a nut shell:

  • Add a small suffix to our easy-deflate module that wraps it in a CompressionStream-compatible interface.
  • Deprecate and remove direct access to the non-CompressionStream globals. Possibly by proxy of deprecating the module itself and re-exposing under a new name (e.g. compression-stream)
  • Make the compression-stream a "skippable" module based on a CompressionStream feature-test (e.g. !!window.CompressionStream). Similar to how we did in the past with the json, dom-level-2 and es5-shim modules. Thus allowing VisualEditor to them on them, and yet cause no download if they are natively supported.

Links:

See also:

Event Timeline

Krinkle moved this task from Backlog to Reported Upstream on the Upstream board.
Krinkle updated the task description. (Show Details)

Change 542281 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] Remove unused 'easy-deflate.inflate' module

https://gerrit.wikimedia.org/r/542281

Change 542281 merged by jenkins-bot:
[mediawiki/core@master] Remove unused 'easy-deflate.inflate' module

https://gerrit.wikimedia.org/r/542281

Krinkle triaged this task as Medium priority.Oct 11 2019, 1:35 AM
Krinkle removed a project: Patch-For-Review.
Krinkle updated the task description. (Show Details)

This has shipped in Chromium in Dec 2019. – https://bugs.chromium.org/p/chromium/issues/detail?id=99909

Interestingly, the web-platform-tests suite for the API also uses Pako library (same as us), to verify its output. This means we can be pretty sure it's compatible :)

https://github.com/web-platform-tests/wpt/blob/4d71665812982ce4e32e74d13c968a4e888fcdf2/compression/compression-stream.tentative.any.js
https://hg.mozilla.org/mozilla-central/rev/6fa5cbdda5e0

This will also make mw.deflate async, so we'll need to update the callers (VE+CX):

https://codesearch.wmcloud.org/search/?q=mw%5C.deflate&i=nope&files=&excludeFiles=&repos=

No on-wiki users:

https://global-search.toolforge.org/?q=mw%5C.deflate&regex=1&namespaces=2%2C4%2C8&title=%28Gadgets-definition%7C.*%5C.%28js%7Ccss%7Cjson%29%29

The native function appears to be about twice as fast, compressing 800k of HTML in 40ms instead of 100ms.

Esanders renamed this task from Use CompressionStream API where available (instead of easy-deflate polyfill) to Use CompressionStream API where available (instead of Pako polyfill).Jul 12 2022, 4:17 PM

@Esanders Yeah, we'll need a deprecation/migration for that. What are your thoughts regarding not exposing a wrapper but migrating directly to the native, without a mw.* replacement? E.g. a polyfill module loaded with skipFunction and then VE would use that native (async) API directly.

I guess it depends on ergonimics. If it's too verbose, we can keep the mediawiki.deflate module around and expose a new async method for convenience and migrate towards that, with the polyfil as a separate module. I'd be open to that, but just wanted to confirm that that's actually your preference.

CompressionStream seems like an interesting concept, but on wiki we generally want to compress to and from strings. (and preferably the kind of strings that can be stored in user preferences and localStorage) CompressionStream can undoubtedly do that, but it seems it won't be trivial. So providing an mw.* to compress (like mw.deflate) and decompress (not available atm) would be very welcome. EditNoticesOnMobile (T312299) could have taken advantage of that. My other script, Bawl, currently uses lz-string and I was planning to make the transition to mw.deflate+pako.inflate. Guess I'll need full pako now, at least until T312720 is resolved.

Jack who built ConvenientDiscussions just subscribed to that task, I'm thinking there'll be interest.

There may be no users of mw.deflate right now but that's hardly surprising as the inflate module isn't provided. I just found out how to inflate so it's possible nobody else knew. Or found it too bothersome/risky.

Change 814853 had a related patch set uploaded (by Esanders; author: Esanders):

[mediawiki/core@master] WIP native compression

https://gerrit.wikimedia.org/r/814853

This task is about using CompressionStream to improve performance. Please don't use it to complain about another task not getting prioritised.

This task is about using CompressionStream to improve performance. Please don't use it to complain about another task not getting prioritised.

Wtf, I'm not complaining and this is completely related, isn't it? They can use the exact same stuff. Krinkle said on T312720 that "Alternatively, I would accept a patch to help resolve T235237".

And why wouldn't you want to try and kill two birds with one stone, or at least, try to make sure the stone you get for one bird will be suitable for the next as well (even if you don't intend to kill it right away) so you don't end up with two stones.

@Esanders Yeah, we'll need a deprecation/migration for that. What are your thoughts regarding not exposing a wrapper but migrating directly to the native, without a mw.* replacement? E.g. a polyfill module loaded with skipFunction and then VE would use that native (async) API directly.

I guess it depends on ergonimics. If it's too verbose, we can keep the mediawiki.deflate module around and expose a new async method for convenience and migrate towards that, with the polyfil as a separate module. I'd be open to that, but just wanted to confirm that that's actually your preference.

If you look at my WIP patch the native stuff is pretty verbose for converting the data into a string for a form (although I can't guarantee our base64 approach is the most efficient), so I would think we'd want to keep the mw.* utilities for now.

For the record:

Patch by @Esanders is merged:

[mediawiki/core] Update pako to 2.0.4

As Pako no longer supports string conversion we have to
do our own in mw.deflate.

https://gerrit.wikimedia.org/r/c/mediawiki/core/+/812909

Also for the record, I've provided a smaller and simpler mw.deflate alternative in T312720#8098113 as well as its mw.inflate counterpart.