Page MenuHomePhabricator

[Spike] Build step, ResourceLoader, or both for Vue.js search?
Closed, ResolvedPublicSpike

Description

There are two ways to process Vue.js templates in MediaWiki: a build step and on-the-fly compilation. Which should we we use for this project? That's what this task decides.

Acceptance criteria

  • A decision to use a build step, on-the-fly compilation, or both is made.
  • At least one new task is created to implement the choice(s) and add Vue.js as a dependency.
  • Documentation on wiki is revised with any learnings.
  • Feedback on this decision is solicited in the developer survey. (This means adding some nicely formed question(s) to the survey task.)

Build step

Summary: the most performant, flexible, and least risky option, but more upfront work.

Pros

  • The most performant byte sizes and rendering. This derisks a major concern for the project as much as possible. Specifically
  • Web has the experience (e.g., Popups, MobileFrontend, and Vector Storybook) to define good patterns for the broader organization.
  • Sharing the bundler configuration may allow some commonization of Web's existing six Webpack configs.
  • Immense flexibility to leverage any tool in the JavaScript ecosystem. A build step, formal or otherwise, enables an extraordinarily useful abstraction between the code written by developers and the bytes shipped to users. A build step is a gateway to the entire wealth of tooling in the JavaScript ecosystem second only to package.json itself.
  • Provides additional flexibility for aggregating and splitting dependencies within projects.
  • There are no limitations on the Vue.js code written. That is, all source will be canonical Vue.js, not MediaWiki-specific Vue.js. This meets the expectations of newcomers and developers interested in using our components in non-MediaWiki contexts.
  • Developer productivity boost via hot module replacement functionality. In @santhosh's words, "Cannot even compare it with the debug=true mode of development speed :-)"
  • Local build steps have historically had excellent production success.
  • SVGs can be optimized by Webpack before shipping (agreed upon internally as an understood expectation).
  • This is a non-binary decision. It's more accurately put ResourceLoader _and_ build step since the outputs of the build step just feed back into ResourceLoader.
  • Additional pros enumerated in "front-end development environment" discussion.

Cons

  • Clumsy asset versioning. Plain and simple: we know this song and dance well but it is a friction until the build step RFC is complete. More specifically, most JavaScript patches will create merge conflicts, we have to make another copy of our dist test script, and our Git history is polluted with build products. This friction hurts newcomers especially.
  • Requires extra configuration by Web to set good patterns for other interested teams.
  • Sharing the bundler configuration requires further additional work.
  • Limited by whatever the MediaWiki ecosystem can provide.
  • Requires additional consideration for sharing dependencies across projects (e.g., runtimes).
  • Code has to be built before changes appear. If you forget or are unable to build the code, you don't get the code.
  • If only a build step is used, no feedback is provided on the on-the-compilation implementation.
  • The build step RFC isn't even done yet.

Example Webpack configuration

{
	"entry": "./resources/skins.vector.js/skin.js",
	"output": {
		"filename": "skins.vector.js"
	},
	"externals": [
		"mediawiki.page.ready"
	]
}

On-the-fly compilation

Summary: less performant and inflexible, but very little upfront work.

Pros

  • Minimal configuration. Build step tooling is always a pain to configure properly but on-the-fly supports very little configuration.
  • Less maintenance. Build-step specific tooling requires periodic upgrades. On-the-fly compilation is part of Core.
  • All code written is most likely to work in any MediaWiki context.
  • This workflow is supported by at least two Foundation developers and may be the mechanism used by gadgets and tools unable to use a build step.
  • The version of Less used must match MediaWiki's version. Everything that works in normal MediaWiki styles works here (e.g., CSSJanus, relative URLs, imports, etc).
  • Possibly the standard MediaWiki way to build Vue.js apps.
  • Additional limitations for on-the-fly compilation may be uncovered during development. That's exactly the kind of thing FAWG should know about sooner rather than later.

Cons

  • All code written is less likely to work in any non-MediaWiki context.
  • Scoped styles are unsupported.
  • What you write is what you ship--No ES6, TypeScript, or language compilation support. At time of writing, this means ES5 only.
  • Vue.js directive shorthands are unsupported @ for v-on and : for v-bind.
  • This tooling is early in development by and for MediaWiki, but not as well tested as Webpack.
  • The version of Less used must match MediaWiki's version (very old).
  • Definitely the non-standard way to build Vue.js apps for the rest of the world.
  • Additional limitations for on-the-fly compilation may be uncovered during development. That hurts product development.
  • If only on-the-fly compilation is used, no feedback is provided on the build step implementation.
  • SVGs must be committed minified or delivered to the client in "debug mode."

Example ResourceLoader configuration

"skins.vector.js": {
    "packageFiles": [
        "resources/skins.vector.js/skin.js",
        "resources/skins.vector.legacy.js/collapsibleTabs.js",
        "resources/skins.vector.legacy.js/vector.js"
    ]
},

References

Conclusion

A build step will be used:

  • Search development will occur in a feature branch to allow build scaffolding to be configured without blocking the rest of the Desktop Improvements Project.
  • Relevant status updates will be communicated on the search update log.
  • Vue.js search will be for Latest Vector only.
  • Conversations have been started with RelEng. Any work with Security and RelEng will be tasked as needed.

Event Timeline

Restricted Application changed the subtype of this task from "Task" to "Spike". · View Herald TranscriptApr 3 2020, 5:09 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Niedzielski renamed this task from [Spike] Build step, ResourceLoader, or both? to [Spike] Build step, ResourceLoader, or both for Vue.js search?.Apr 5 2020, 4:23 PM

Since the library is compiled, we have a blob of JavaScript we have to include into ResourceLoader some how. As far as I know, that means Vector will require a "resources/dist"-like directory to hold the dependencies at least until a build step is available. In pre-Webpack MobileFrontend, for example, we called it "libs/" and manually copied hogan.js in there. In post-Webpack Popups and MobileFrontend, for example, that's been resources/dist and the Webpack managed manages the bundling, splitting, transpilation, etc.

So this task is: how should we manage "resources/dist/" or "libs/" until a build step is available? I think there are these options:

  1. File copy:
    1. NPM package. We could write a script that copied the WVUI build product from node_modules/@wikimedia/wvui/dist/*.js to resources/dist.
    2. Git submodule. Add a WVUI Git submodule. This means the build product can be referenced by file path, assuming cross-repo paths don't require a custom CI configuration.
    3. Manual file copy. After each WVUI release, copy the file by hand.
    4. Something else?
  2. Webpack / Vue CLI:
    1. NPM package. A standard NPM dependency.
    2. Git submodule. NPM is file path aware and, I think, we could reference the WVUI library by file path. This might require special CI configuration though.
    3. Something else?
  3. Something else?

One thing I dislike about any of the file copy approaches is the need reinvent a "poor man's NPM" for dependency management.

For option 2.a, I've posted a minimal example Webpack configuration into the task. If you type npm install @wikimedia/wvui and then require('@wikimedia/wvui') in skins.js, the dependency will be automatically included in the bundle. It's not a zero config but it's not far from it. The more specialization you want, the more config you need but there's opportunities to make that configuration common and share it.

If we go with option 2, we should probably consider Rollup, especially if we ever plan to leverage ES2015 modules in browser.

Rich Harris has a nice post where he summarizes "Use webpack for apps, and Rollup for libraries" and we are definitely creating a library. I know we have talked about code splitting a lot but I honestly am yet to see any concrete use cases for this in the current ecosystem unless we are planning to throw out ResourceLoader entirely. I'd love to see some if anybody has some practical examples.

ES2015 modules

Thanks @Jdlrobson! This is a known issue but appears to have traction.

consider Rollup

There's some private discourse on zero-config tools here like rollup.js. I think Webpack is pretty close to zero-config (working example shown above in the task description). It becomes more non-zero the more specialization you want. I'll also add that Vue CLI and Storybook both use Webpack under the hood.

I know we have talked about code splitting a lot but I honestly am yet to see any concrete use cases for this in the current ecosystem unless we are planning to throw out ResourceLoader entirely.

Code splitting can be done manually as is the case for ResourceLoader. However, there's great value in being able to coalesce common blobs of code shared among different entries. Let's not forget mobile.common in MobileFrontend! I think this functionality would be especially useful as more projects migrate to Vue.js.

Niedzielski changed the task status from Open to Stalled.Jun 17 2020, 4:48 PM

Could one expand: “The most performant byte sizes and rendering. This derisks a major concern for the project as much as possible.” What risk do we have when relying on RL? How big is it, where are the byte size comparisons for this? Is this a known shortcoming? Asking as this should be flagged elsewhere (MediaWiki-ResourceLoader) if it hasn't already been.

@Volker_E, bandwidth and CPU consumption. According to the Vue.js docs, "...runtime-only builds are roughly 30% lighter-weight than their full-build counterparts, you should use it whenever you can."

The most performant byte sizes and rendering. This derisks a major concern for the project as much as possible.

While this is true for now, will this be true on the long term? For example, let's say Echo wants to use Vue. How is this extension going to use Vue without shipping an additional copy of Vue on top of the existing desktop one? Won't we end up with two versions of Vue, possibly with differing versions? Sounds like we'd be prohibiting any extension integrating desktop from using Vue without a plan there.

While this is true for now, will this be true on the long term? For example, let's say Echo wants to use Vue. How is this extension going to use Vue without shipping an additional copy of Vue on top of the existing desktop one? Won't we end up with two versions of Vue, possibly with differing versions? Sounds like we'd be prohibiting any extension integrating desktop from using Vue without a plan there.

@Jdlrobson, yes. If Echo wants to use the build step, it can stick with the runtime. If it wants to compile on the clients instead, it can load the compiler in addition to the existing runtime which is discussed in T252348. Versioning concerns are an impetus for T252348 and why I hope the runtime will land in Core.

I think the conclusion makes a lot of sense. We've been asked to do a case study on Vue so I think reviewing with a build step is going to be highly informative. There is plenty of case study material for without it and our learnings are going to be important for the RFC. Thanks for making a note to start those conversations with RelEng and Performance ( I wonder if gitlab can help with this in some way).

Using a branch makes a lot of sense as it keeps the case study separate from the desktop improvements work and deployment. We'll want to regularly rebase it against master and make sure we've carved out a merge conflict free space to avoid any hiccups. Given the approach in T250851 I'm expecting this to be in its own RL module so this shouldn't be a problem.

Shall we move this to sign off and talk about it on monday in our SHDT meeting?

Jdlrobson claimed this task.

We discussed on Monday resolving per T249350#6291829.