Change Details

The latest version of all major browsers support C/C++/etc code compiled to WebAssembly ("wasm") format, in addition to the "asm.js" style of compilation targeting a subset of JavaScript directly. [Work in progress, please feel free to suggest updates] The first library that MediaWiki uses to support WebAssembly is ogv.js, a codec & playback library which we use for playing Ogg and WebM files on Safari, IE, and Edge browsers. ogv.js is packaged with the TimedMediaHandler extension, and we've used its existing asm.js mode for about two years now. I'd like to enable the WebAssembly mode in production for faster compilation/load times, but want to double-check that we've established some best practices for WebAssembly usage first. == Tech overview == * [asm.js](http://asmjs.org/) is a subset of JavaScript which is used for outputting compiled code in a way that the browser can recompile efficiently. * [WebAssembly](http://webassembly.org/) is a compact binary format for a device-independent bytecode, roughly equivalent to asm.js in capabilities and usage patterns but with a smaller footprint and quicker parsing/compilation times. * [emscripten](http://emscripten.org/) is a popular compiler targeting both these platforms. A library using WebAssembly or asm.js compiled code will generally have three parts: * client JS code or wrapper library * compiler-generated JS "glue code" * asm.js or .wasm compiled code The asm.js or .wasm compiled code is built from C/C++ or other source files with clang and the emscripten compiler; this process may be as complex as any other native code build or may be a few simple command invocations. == Security == The compiled code has access only to a chunk of linear memory and whatever JavaScript functions are passed into it by the glue code, which may be very few or very many -- so security surface varies. Compiled code may be a simple "headless" library, or may manage a WebGL context for visualization, input events, etc. Best practice: when checking a wasm/asm.js library for security, check the 'contract' in the glue code to determine what needs to be examined more closely Open questions: safety for library releases etc? == Compatibility == asm.js code output runs in all of our level-A compatibility browsers, requiring only Typed Arrays -- even IE 11! All major browser engines in their latest versions support WebAssembly as well, which can be detected at runtime. Modules that package WebAssembly code should generally include an asm.js build as well, and select the one to load at runtime -- asm.js takes longer to load and compile. Best practice: include either asm.js only, or both wasm & asm.js Open questions: none? == Source vs binary check-ins == Compiling code to WebAssembly or asm.js is like compiling native C/C++/rust/whatever code, not like minifying JavaScript. This means it's not suitable for runtime transformations via ResourceLoader. Best practice: C/C++ source lives in separate library, .wasm+.js "binaries" checked in to MW core or ext like any library Open questions: should we use package managers instead? (npm/composer/something) == Loading asm.js and wasm code == asm.js code may have trouble running through ResourceLoader's minification process -- TimedMediaHandler currently loads ogv.js's front-end JavaScript through ResourceLoader and loads the codec payloads directly as asset files. .wasm code blobs are loaded at runtime as assets, which may require passing wgExtensionAssetPath etc into the initializing JavaScript code. Best practice: treat large asm.js blobs and .wasm blobs as raw assets outside RL; pass the proper URLs into initializer code Open question: can/should we improve this? == Credit and licensing == Licenses of compiled code may require offer of source (GPLv2) or copyright notices (BSD). What's best practice to include these in Special:Version? Best practice: unknown Open questions: best way to include credits? == Action items == * talk through any further issues * look at the npm integration issue? * find appropriate part of contributor guidelines to update * update it

The latest version of all major browsers support C/C++/etc code compiled to WebAssembly ("wasm") format, in addition to the "asm.js" style of compilation targeting a subset of JavaScript directly. [Work in progress, please feel free to suggest updates] The first library that MediaWiki uses to support WebAssembly is ogv.js, a codec & playback library which we use for playing Ogg and WebM files on Safari, IE, and Edge browsers. ogv.js is packaged with the TimedMediaHandler extension, and we've used its existing asm.js mode for about two years now. I'd like to enable the WebAssembly mode in production for faster compilation/load times, but want to double-check that we've established some best practices for WebAssembly usage first. == Tech overview == * [asm.js](http://asmjs.org/) is a subset of JavaScript which is used for outputting compiled code in a way that the browser can recompile efficiently. * [WebAssembly](http://webassembly.org/) is a compact binary format for a device-independent bytecode, roughly equivalent to asm.js in capabilities and usage patterns but with a smaller footprint and quicker parsing/compilation times. * [emscripten](http://emscripten.org/) is a popular compiler targeting both these platforms. A library using WebAssembly or asm.js compiled code will generally have three parts: * client JS code or wrapper library * compiler-generated JS "glue code" * asm.js or .wasm compiled code The asm.js or .wasm compiled code is built from C/C++ or other source files with clang and the emscripten compiler; this process may be as complex as any other native code build or may be a few simple command invocations. == Security == The compiled code has access only to a chunk of linear memory and whatever JavaScript functions are passed into it by the glue code, which may be very few or very many -- so security surface varies. Compiled code may be a simple "headless" library, or may manage a WebGL context for visualization, input events, etc. Note that the Spectre speculative-execution data exposure timing vulnerability could be exploited with both wasm and js code given sufficiently precise timers. This is being actively mitigated by browser makers with a combination of timing source fuzzing and reworking their JIT compilers to produce safer code. Best practice: when checking a wasm/asm.js library for security, check the 'contract' in the glue code to determine what needs to be examined more closely Open questions: safety for library releases etc? == Compatibility == asm.js code output runs in all of our level-A compatibility browsers, requiring only Typed Arrays -- even IE 11! All major browser engines in their latest versions support WebAssembly as well, which can be detected at runtime. (Code built with recent emscripten versions requires the LEGACY_VM_SUPPORT option for IE 11 compatibility.) Modules that package WebAssembly code should generally include an asm.js build as well, and select the one to load at runtime -- asm.js takes longer to load and compile. Best practice: include either asm.js only, or both wasm & asm.js Open questions: none? == Source vs binary check-ins == Compiling code to WebAssembly or asm.js is like compiling native C/C++/rust/whatever code, not like minifying JavaScript. This means it's not suitable for runtime transformations via ResourceLoader. Best practice: C/C++ source lives in separate library, .wasm+.js "binaries" checked in to MW core or ext like any library. Publishing via npm and having a local script to pull the updates from node_modules into source is probably best for now. Open questions: Is there a better way to handle JS+assets from package manager sources? (cf T107561) == Loading asm.js and wasm code == asm.js code may have trouble running through ResourceLoader's minification process -- TimedMediaHandler currently loads ogv.js's front-end JavaScript through ResourceLoader and loads the codec payloads directly as asset files. .wasm code blobs are loaded at runtime as assets, which may require passing wgExtensionAssetPath etc into the initializing JavaScript code. Best practice: treat large asm.js blobs and .wasm blobs as raw assets outside RL; pass the proper URLs into initializer code Open question: can/should we improve this? == Credit and licensing == Licenses of compiled code may require offer of source (GPLv2) or copyright notices (BSD). What's best practice to include these in Special:Version? Best practice: treat like other JS libraries Open questions: automation? == Action items == * talk through any further issues * look at the npm integration issue? * find appropriate part of contributor guidelines to update * update it

The latest version of all major browsers support C/C++/etc code compiled to WebAssembly ("wasm") format, in addition to the "asm.js" style of compilation targeting a subset of JavaScript directly. [Work in progress, please feel free to suggest updates] The first library that MediaWiki uses to support WebAssembly is ogv.js, a codec & playback library which we use for playing Ogg and WebM files on Safari, IE, and Edge browsers. ogv.js is packaged with the TimedMediaHandler extension, and we've used its existing asm.js mode for about two years now. I'd like to enable the WebAssembly mode in production for faster compilation/load times, but want to double-check that we've established some best practices for WebAssembly usage first. == Tech overview == * [asm.js](http://asmjs.org/) is a subset of JavaScript which is used for outputting compiled code in a way that the browser can recompile efficiently. * [WebAssembly](http://webassembly.org/) is a compact binary format for a device-independent bytecode, roughly equivalent to asm.js in capabilities and usage patterns but with a smaller footprint and quicker parsing/compilation times. * [emscripten](http://emscripten.org/) is a popular compiler targeting both these platforms. A library using WebAssembly or asm.js compiled code will generally have three parts: * client JS code or wrapper library * compiler-generated JS "glue code" * asm.js or .wasm compiled code The asm.js or .wasm compiled code is built from C/C++ or other source files with clang and the emscripten compiler; this process may be as complex as any other native code build or may be a few simple command invocations. == Security == The compiled code has access only to a chunk of linear memory and whatever JavaScript functions are passed into it by the glue code, which may be very few or very many -- so security surface varies. Compiled code may be a simple "headless" library, or may manage a WebGL context for visualization, input events, etc. Note that the Spectre speculative-execution data exposure timing vulnerability could be exploited with both wasm and js code given sufficiently precise timers. This is being actively mitigated by browser makers with a combination of timing source fuzzing and reworking their JIT compilers to produce safer code. Best practice: when checking a wasm/asm.js library for security, check the 'contract' in the glue code to determine what needs to be examined more closely Open questions: safety for library releases etc? == Compatibility == asm.js code output runs in all of our level-A compatibility browsers, requiring only Typed Arrays -- even IE 11! All major browser engines in their latest versions support WebAssembly as well, which can be detected at runtime. (Code built with recent emscripten versions requires the LEGACY_VM_SUPPORT option for IE 11 compatibility.) Modules that package WebAssembly code should generally include an asm.js build as well, and select the one to load at runtime -- asm.js takes longer to load and compile. Best practice: include either asm.js only, or both wasm & asm.js Open questions: none? == Source vs binary check-ins == Compiling code to WebAssembly or asm.js is like compiling native C/C++/rust/whatever code, not like minifying JavaScript. This means it's not suitable for runtime transformations via ResourceLoader. Best practice: C/C++ source lives in separate library, .wasm+.js "binaries" checked in to MW core or ext like any library. Publishing via npm and having a local script to pull the updates from node_modules into source is probably best for now. Open questions: should we useIs there a better way to handle JS+assets from package managers instead sources? (npm/composer/somethingcf T107561) == Loading asm.js and wasm code == asm.js code may have trouble running through ResourceLoader's minification process -- TimedMediaHandler currently loads ogv.js's front-end JavaScript through ResourceLoader and loads the codec payloads directly as asset files. .wasm code blobs are loaded at runtime as assets, which may require passing wgExtensionAssetPath etc into the initializing JavaScript code. Best practice: treat large asm.js blobs and .wasm blobs as raw assets outside RL; pass the proper URLs into initializer code Open question: can/should we improve this? == Credit and licensing == Licenses of compiled code may require offer of source (GPLv2) or copyright notices (BSD). What's best practice to include these in Special:Version? Best practice: unknowntreat like other JS libraries Open questions: best way to include creditsautomation? == Action items == * talk through any further issues * look at the npm integration issue? * find appropriate part of contributor guidelines to update * update it