We talked about this in standup and attempted to estimate.

We decided we'd prefer a solution that uses ResourceLoader URIs to check bundlesizes rather than builds assets itself using lessc (as we did in Minerva)

We'd like to investigate how possible that is before estimating.

From my perspective, given the tightly coupled Vue.js integration, it would be best if this task were completed before the search experience work starts.

We decided we'd prefer a solution that uses ResourceLoader URIs to check bundlesizes rather than builds assets itself using lessc (as we did in Minerva)

I missed the discussion last week but I think using Webpack for the JavaScript bundlesize tests would be useful both for the purposes of measuring performance as well as all the other functionality it provides, notably giving us the flexibility to compile _and_ minify the Vue.js + search JavaScript bundle.

I am still not convinced that Vue code will live inside Vector (it may live in core). If we want to add webpack as part of this change I would strongly suggest punting this until work on search begins. Setting up webpack is a task in itself.

If we want to use ResourceLoader URIs we'll need to make sure an environment variable exists pointing to a mediawiki instance and that environment variable will need to be setup manually as it can vary between users (I use localhost:8888 for example). It looks like wmf-quibble-core-vendor-mysql-php72-docker has one but you can't add jobs to that like you can in npm test AFAICS.

With these two things in mind I suggest either taking the simple tried and tested Minerva approach of lessc + bundlesize or punting this task until later. Moving to upcoming to have that conversation.

Per T205129#4671988, mw.inspect() may not be the best evaluation for these metrics. Maybe this is something Fresnel can help with though. /cc @Jdrewniak

This task was originally supposed to evaluate JavaScript sizing differences for the search experience _and_ any of its dependencies (e.g., the Vue.js compiler or a component library), but there are other important metrics to measure such as styling size and rendering performance. The hope was to avoid unexpected changes in performance, most especially for enterprising Vue.js work. I think perspectives from multiple tools would help us have higher confidence in the changes we make.

Maybe the Performance-Team has some insights. We're going to be developing a new search experience in Vector using Vue.js at some point, possibly soon. Do you have any recommendations on how best to measure the performance impact of each patch? How should we minimize risk for new patches from multiple contributors on and off the Web team?

I was thinking it would be nice to at least understand the JavaScript, template, styles, Vue.js compiler, and any dependencies. Comparing different loading strategies like static ResourceLoader module dependency vs deferring loading after initial render would also be valuable. Do you consider Fresnel adequate or should we explore additional options?

Inside Quibble/CI and most mw-dev environments there are MW_* environment variables available. These are used by Selenium and QUnit jobs for example, which means if you're running either of those from the command-line locally today, you have them set up (e.g. saved in your bashrc file, or automatically by Docker/Vagrant).

A few lines of bash or Node code to make a request to load.php would get you there. For example:

#!/bin/bash -eu

curl -fsS "$MW_SERVER/$MW_SCRIPT_PATH/load.php?lang=en&modules=skins.vector.styles.legacy&only=styles" > /tmp/output.css
wc -c /tmp/output.css
#> 33745 bytes
gzip -9 -c /tmp/output.css | wc -c
#> 7511 bytes

Change 586413 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/Vector@master] Check and enforce bundlesizes in Vector

https://gerrit.wikimedia.org/r/586413

Thanks for the protip @Krinkle I hadn't realised those globals were available here.

@Krinkle I'm seeing "curl: (3) <url> malformed" which suggests those globals are not available (works fine locally). Are you sure they are available in npm test?

In T244276#6033660, @Jdlrobson wrote:

@Krinkle I'm seeing "curl: (3) <url> malformed" which suggests those globals are not available (works fine locally). Are you sure they are available in npm test?

No, the basic Node.js pipeline just clones the repo and runs npm test. It's not that the globals are missing, the MediaWiki install command is skipped there entirely. As a quick hack you can try adding it to packages.scripts.selenium or (if already set) to packages.scripts.postselenium - to confirm that it does indeed work.

Hacks aside, you'll want to add this npm run command to the quibble job for the Vector repo. If the list of commands Quibble ran was maintained in a simple .ci.yaml file that would be easy to do, but that is not yet the case. I can take look but I don't know off-hand where to add it. RelEng would know best. It's probably somewhere in integration/config.

Change 586413 abandoned by Jdlrobson:
Check and enforce bundlesizes in Vector

Reason:
My goal here was to determine if the idea works in npm test (which it doesnt).

I'll abandon this to be clearer that I'm not planning to work on this further. We can talk about writing in Node.js during estimation but I suspect we will be limited by a generic solution in the CI pipeline that lives outside this repo.

https://gerrit.wikimedia.org/r/586413

Can we revise the title of this so it's not linked to Vue.js search? We need this for everything else too :)

Change 601758 had a related patch set uploaded (by Jdrewniak; owner: Jdrewniak):
[mediawiki/skins/Vector@master] Add bundlesize test for ResourceLoader modules.

https://gerrit.wikimedia.org/r/601758

#!/bin/bash -eu
curl -fsS "$MW_SERVER/$MW_SCRIPT_PATH/load.php?lang=en&modules=skins.vector.styles.legacy&only=styles" > /tmp/output.css
gzip -9 -c /tmp/output.css | wc -c
#> 7511 bytes

So the patch above basically does that but with 50 lines of Node.js as well as the bundlesize library. I figure it's worth it to maintain consistency with our other repos which also use bundlesize.

Change 602780 had a related patch set uploaded (by Jdrewniak; owner: Jdrewniak):
[mediawiki/skins/Vector@master] [POC] Alternative to 9c55d62276e8c7d7c6bf4f43524c32f1eff0e6d3

https://gerrit.wikimedia.org/r/602780

As a quick hack you can try adding it to packages.scripts.selenium or (if already set) to packages.scripts.postselenium - to confirm that it does indeed work.

@Krinkle thanks for the tip, but it looks like the Vector repo doesn't have any selenium pipeline set up right now (since that doesn't run :( )

So I'm hope someone from Release-Engineering-Team can help us out here. tldr; We want to set up a generic Selenium job for the Vector skin that let's us run npm run selenium or as @Krinkle suggests postselenium.

Change 601758 abandoned by Jdrewniak:
Add bundlesize test for ResourceLoader modules.

https://gerrit.wikimedia.org/r/601758

I've split out the work of adding this job to CI as a separate task: T255149 since that will require some of @hashar expertise.

Also of interest to the team, as @hashar informed me, WMDE has just released a similar tool for measuring their bundle sizes, described in T253204.

https://doc.wikimedia.org/Wikibase/master/js/tainted-ref-dist-size/

Pretty cool!

Also, I did get the script to run in CI, via npm run selenium-test.

tainted-ref-dist-size

Very cool! Is this Gerrit-only or is there some NPM package we can use locally?

Change 602381 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/Vector@master] Pin bundlesizes to their exact values

https://gerrit.wikimedia.org/r/602381

Change 602780 merged by jenkins-bot:
[mediawiki/skins/Vector@master] Add bundlesize test for ResourceLoader modules.

https://gerrit.wikimedia.org/r/602780

Change 602381 merged by jenkins-bot:
[mediawiki/skins/Vector@master] Pin bundlesizes to their exact values

https://gerrit.wikimedia.org/r/602381

@Jdrewniak, I'm bouncing this back to needs more work to signal consideration of the following: 1) try to update the changelog with changes so this doesn't need to happen on release 2) it would be nice if bundlesize reports could be generated with the rest of the docs (e.g., npm -s run test:size|grep dist > docs/minGzipBundleSize.txt) 3) consider updating the readme with details on the bundlesize reports / tests.

In T244276#6244549, @Niedzielski wrote:

2. it would be nice if bundlesize reports could be generated with the rest of the docs (e.g., npm -s run test:size|grep dist > docs/minGzipBundleSize.txt)

A script to update bundlesize.config.json would be helpful as well. Now I do it by hand when the CI test rejects the patch... 1 step later than I should.

A script to update bundlesize.config.json would be helpful as well. Now I do it by hand when the CI test rejects the patch... 1 step later than I should.

I agree but I think this change would be best upstream in the bundlesize tool itself. E.g., something like Jest's -u flag for updating snapshots.

In T244276#6244594, @Niedzielski wrote:

I think this change would be best upstream in the bundlesize tool itself. E.g., something like Jest's -u flag for updating snapshots.

Yes, would be better. I've found only this - slightly related - feature request: #322.

A note: I've had to adjust bundlesize.config.json a few times in a day and I'm close to getting merge conflicts on it...
While the modern layout is in development - not enabled on any wikis -, can we go with slightly more relaxed limits, like +0.2KiB rounded up?

In T244276#6255192, @Demian wrote:

A note: I've had to adjust bundlesize.config.json a few times in a day and I'm close to getting merge conflicts on it...
While the modern layout is in development - not enabled on any wikis -, can we go with slightly more relaxed limits, like +0.2KiB rounded up?

This is by design. The precision is important and we be mindful about bytes shipped at all time and how much our changes are impacting it. I think merge conflicts are a small price to pay.

Yesterday I had the first merge conflict in bundlesize and today this:
FAIL skins.vector.styles: 8.2KB > maxSize 8.2KB (gzip)
Comparing floating points...