Page MenuHomePhabricator

[EPIC] Automate asset bundling in MobileFrontend
Closed, ResolvedPublic

Description

Documentation on wiki

https://m.mediawiki.org/wiki/Reading/Web/Projects/Invest_in_the_MobileFrontend_%26_MinervaNeue_frontend_architecture#Automate_asset_bundling

What

Use a Node.js based bundler (webpack) to compile the JavaScript files into different entry points automatically based only on source code import statements. These bundled files will then be given to ResourceLoader for serving.

This approach is the same one taken in Extension:Popups (Page Previews), which has proven to be an effective strategy.

We will also use UglifyJS for minifying our JavaScript files for production, which should give us approximately 25% savings on file sizes for the mobile website.

Progress

There are ~80 JavaScript files in MobileFrontend, in 58 modules.

We will go module by module, file by file, migrating the file to the src/ folder like in Extension:Popups, adapting the imports/exports, and moving the QUnit test files to node-qunit if possible/easy.

Progress can be measured by looking at the amount of files and lines of code migrated to the src/ folder.

Measuring success

We will be successful if:

  • There is a reduction in the size of the JS assets served to clients due to better minimization
  • There is a big reduction on the amount of manual JSON configuration needed for sorting the JS files (.ResourceModules on extension.json)
  • The performance of MediaWiki locally on development is better due to the decreased load on ResourceLoader

AC

  • We have documented the client bundle sizes served, the amount of manual configuration for scripts in .ResourceModules extension.json, for further analysis after the project ends (see T165036 for an example which includes bundlesize checks, Webpack performance hints, and ADR and blog post docs)
  • There is a build setup with npm scripts start and build that bundles the assets and outputs them to resources/dist
  • CI checks on every commit that the built assets committed in the repo are up to date with the committed sources (like in Extension:Popups-an experiment is underway to _not_ version artifacts)
  • The build process is configured to use UglifyJS when outputting production assets
  • The development build process includes a script to monitor and automatically rebuild debug assets (see Popups)
  • The build assets are properly configured to be served by ResourceLoader from MediaWiki (see Popups)
  • The sources in resources/ have been migrated to src/ ~~~using EcmaScript modules syntax~~~ [descoped]
  • and babel for transpilation to ES5
  • There is a linting step for verifying that the production bundles in resources/dist are ES5 compliant (see Popup's ES5 ESLint config or this eslint-plugin-compat spike)
  • The built assets are split automatically to have common modules of bundles based on the entry points, and configured correctly to be performant for our most common cases
  • We measure our success metrics after the work is finished and compare and report with the initial metrics

Sign off steps

  • Report success metrics!

Related Objects

Event Timeline

Jhernandez updated the task description. (Show Details)May 24 2018, 12:04 PM
ovasileva renamed this task from Automate asset bundling in MobileFrontend/Minerva to [EPIC] Automate asset bundling in MobileFrontend/Minerva.Jun 4 2018, 10:07 AM
Jdlrobson moved this task from Backlog to New adventures on the MinervaNeue board.Jun 13 2018, 3:55 PM
Vvjjkkii renamed this task from [EPIC] Automate asset bundling in MobileFrontend/Minerva to gdcaaaaaaa.Jul 1 2018, 1:08 AM
Vvjjkkii triaged this task as High priority.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
CommunityTechBot renamed this task from gdcaaaaaaa to [EPIC] Automate asset bundling in MobileFrontend/Minerva.Jul 2 2018, 3:52 PM
CommunityTechBot raised the priority of this task from High to Needs Triage.
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot added a subscriber: Aklapper.
Jdlrobson triaged this task as High priority.Jul 9 2018, 4:30 PM
Jdlrobson moved this task from Inbox to Next up on the User-Jdlrobson board.Jul 31 2018, 1:58 AM
Jdlrobson changed the status of subtask T202746: Enable babel transpiling in MobileFrontend from Open to Stalled.Oct 5 2018, 5:49 PM
Jdlrobson updated the task description. (Show Details)Oct 10 2018, 8:22 PM
Jdlrobson updated the task description. (Show Details)Oct 23 2018, 7:31 PM
Jdlrobson changed the status of subtask T202746: Enable babel transpiling in MobileFrontend from Stalled to Open.Jan 2 2019, 5:00 PM
Jdlrobson renamed this task from [EPIC] Automate asset bundling in MobileFrontend/Minerva to [EPIC] Automate asset bundling in MobileFrontend.Jan 18 2019, 12:54 AM
Jdlrobson added a subscriber: Jdlrobson.

^ Minerva asset bundling will be covered by T171000

Jdlrobson updated the task description. (Show Details)
Jdlrobson updated the task description. (Show Details)
Jdlrobson updated the task description. (Show Details)Feb 12 2019, 12:34 AM

Report success metrics!

At the start of this project, MobileFrontend had 86 JavaScript files. This was reduced to 19 files built via Webpack from 101 source files. The increase in source files reflects the teams ability to embrace Webpack and separate code responsibilities more effortlessly.

44 QUnit tests were updated so that the could be run from the command line. This allowed us for the first time to see code coverage of the entire MobileFrontend extension. 15 new test files were added thanks to refactorings and improvements to test coverage.

At the start of the project (commit d518689) there were 57 ResourceLoader modules insiding MobileFrontend. 24 ResourceLoader modules were removed in the project leaving 34. This is important as it reduces overhead in the startup module (and impacts performance!). Similar changes are planned in Minerva and various other opportunities exist in MobileFrontend.

During the webpack porting project, we increased code coverage in a few mobile.startup modules. Despite this small effort, upon completing the Webpack migration code coverage was around the 53% mark, specifically:

  • 53.13% of our statements were covered
  • 46.84% of all branches were covered
  • 53.67% of all functions were covered
  • 53.13% of all lines were covered

Given code coverage was purposely increased slightly during this migration, the initial code coverage was likely much less, if we ignore file size and take the average % of all the columns in the table above, the average file featured code coverage with the following characteristics:

  • 50.38% of all our statements were covered
  • 39.2% of all branches were covered
  • 45.2% of all functions were covered
  • 49.4% of all our lines were covered

Most alarmingly of all, 45 of the 81 files had 0% coverage.

We are now able to enforce coverage in the repository meaning that code coverage could only get better.

Good work all!