Page MenuHomePhabricator

Migrate WVUI library to Rollup or Vite
Closed, DeclinedPublic

Description

Replace Webpack with Rollup as the module bundler for WVUI.

Event Timeline

@egardner, While rollup is a better choice here, one major drawback to resolve here is lack of Hot Module Replacement that has negative impact on developer experience and speed. There is no good solution of HMR in rollup since it is just bundler. There were some attempts like Nollup, but recently Vite solved this issue. Vite internally uses rollup for bundling, but does extra magic for fast HMR.

Is it a good idea to explore Vite? We get all benefits of Rollup, plus HMR. Vite is very new to this space though.

@santhosh We are on the same page here. I think that Vite gives us the best of both worlds – simpler build tooling with features like HMR that developers are accustomed to getting from Webpack. In particular Vite's "library mode" feature seems ideally suited for what we are doing here.

We're still exploring options, and whether we use Vite for WVUI may depend on a few other factors (like: can we drop IE11 support and migrate VWUI to Vue 3 now, before we start using it in other projects). Vite in library mode can be configured to emit TypeScript definitions, so that is a plus.

The other thing we'd need to figure out would be what to do with tools like Storybook. Would we keep Storybook (which requires Webpack), perhaps moving it into optionalDependencies to avoid downloading it in some contexts? Would we migrate to a framework-agnostic design system tool like Pattern Lab?

@egardner, Thanks.

Perhaps we need to work on frontend build system first and then apply that to WVUI? or generalize the requirements before we pick one for WVUI? Or do you think the bundler for WVUI can be independent of MW frontend build tooling? I think it is possible technically, but might not be wise choice for a coherant frontend architecture.

We need to think about how a bundler will work with our core+extensions+skins system and resource loader. I am not up to date with the current state of planning from Vue migration team about this(I was on extended leave). I would imagine we need a flexible, modular bundler that can produce assets from extension repos that can be integrated with MediaWiki. In that regard, I see Vite's manifest.json with HMR, SSR, library, fast bundler as good promise. But nothing readymade here - we will need to work on connecting mutliple dots, analysing gaps and opportunities.

I won't worry much about storybook right now. The design system decision is not limited to chosing storybook or alternatives IMHO. It need a wider perspective connecting our branding. I wrote some thoughts here

egardner renamed this task from Migrate WVUI library to Rollup to Migrate WVUI library to Rollup or Vite.Feb 24 2021, 3:54 PM

We know that Rollup builds are readable, auditable, faster to generate, and smaller in size.

One new data point for me from @nolan is that they also avoid the kind of indirect "bundler" syntax that makes bundles significantly slower to compile and execute for the browser's javascript engines. This is especially important now that CPU performance, battery saving, and energy cost are increasingly more the bottleneck for performance and user satisfaction (instead of compressed bandwidth transfer size, which still matters but seems no longer more important than time/energy to process that payload, when all else is equal). In this case it wins on both accounts, but it's worth mentioning because if rollup were merely smaller in output, that could actually be a negative point (e.g. T49437#6044157, etc.).

https://channel9.msdn.com/Blogs/msedgedev/nolanlaw-web-perf-crisis

2017-nolanlaw-rollup-webpack.png (495×1 px, 124 KB)

Change 694972 had a related patch set uploaded (by Eric Gardner; author: Eric Gardner):

[wvui@master] [WIP] Bundle using Rollup

https://gerrit.wikimedia.org/r/694972

Quick update on this: I've opened a patch that ports WVUI over to Rollup for builds. So far here's what's working:

  • Vue / TS compilation works
  • Typescript definitions are being emitted (via rollup-plugin-typescript2)
  • Support for multiple output formats (umd, commonjs, and a new esm build)
  • Support for multiple bundles with different entry points (the main wvui bundle, the limited wvui-search bundle, the icon bundle, etc)
  • Stand-alone CSS files are emitted per bundle
  • Jest unit tests work fine
  • No changes to project tsconfig, linter settings, etc
  • Minification via terser keeps the built files same size or smaller as before

Things that are not currently working:

  • Storybook (which depends on Webpack)
  • Still need to figure out how to generate proper sourcemaps (do people use these for WVUI debugging?)

In my opinion there are several advantages to Rollup over Webpack here. Rollup configuration is much simpler and fewer packages are required for the build, which means there are fewer things that can go wrong. It's also much easier to define different bundle settings (just export another object from the config with the desired settings). Finally, Rollup provides best-in-class support for dead code elimination (aka "tree-shaking"), which will be useful for keeping bundle sizes small for things like the dedicated wvui-search distribution.

The single biggest barrier to porting things over now is lack of Storybook support – this is an important tool and we don't want to have to abandon it.

However, there is some good news on this front as of ~2 months ago: https://storybook.js.org/blog/storybook-for-webpack-5/. Storybook has recently added the ability to define the "builder" that should be used to build the preview application that runs in the browser. The immediate reason for this feature was to allow developers to choose between different major versions of Webpack, but it also opens the door for other build tools like ESBuild and Vite.

The Vite use-case is especially interesting here because Vite is basically Rollup with some improved developer tooling bolted on top to provide HMR. And it looks like someone has already started working on a Vite "builder" for Storybook: https://github.com/eirslett/storybook-builder-vite

Now that I've gotten WVUI mostly working in Rollup, I'm going to see if I can do the same thing with Vite (which supports most Rollup options and plugins), with the added benefit of getting Storybook support without relying on Webpack. 🤞

Will report back when I have more to share.

egardner moved this task from Backlog to Doing on the WVUI board.

Change 698012 had a related patch set uploaded (by Eric Gardner; author: Eric Gardner):

[wvui@master] [WIP] Build WVUI with Vite

https://gerrit.wikimedia.org/r/698012

The single biggest barrier to porting things over now is lack of Storybook support – this is an important tool and we don't want to have to abandon it.

One thing I did wonder, if we are concerned about using storybook for code that doesn't run on our production wikis, but haven't explored, is whether it would make sense to have a single storybook repo for all our extensions, skins inside core or a separate extension. The repo could scan a standardized directory structure e.g.

module.exports = {
        stories: [
                '/extensions/**/stories/**/*.stories.js',
                '/skins/**/stories/**/*.stories.js'
        ]
};

This would mean Webpack is limited to one repo, and would have the benefit of allowing us to display all our components built in Vue.js. It would also mean only having to maintain one storybook install. Potentially might be a pain for development as would need to use a different install.

We'd need some kind of clever cronjob to make sure it reflects current code.

One thing I did wonder, if we are concerned about using storybook for code that doesn't run on our production wikis, but haven't explored, is whether it would make sense to have a single storybook repo for all our extensions, skins inside core or a separate extension.

This is something that I've been thinking about too. The advantage would be removing some overhead from WVUI itself (fewer dependencies required in the repo, easier to get things running locally, etc).

However, doing this will make it more complicated to use Storybook as a tool during component development. It would still be useful for public documentation purposes (display all the merged components and how to use them, etc). Are folks relying on Storybook to help with the initial development of new components (similar to how we use unit tests)? Or do stories usually get written after the fact. @AnneT and @Catrope would be curious as to your thoughts here especially.

Of course, this problem might be mitigated if we go with Vite, because that would give developers the ability to use a demo HTML page (which would also provide HMR during development). In some ways I could see this being easier to work with during the development process – you'd just write normal Vue code (instead of specialized Story syntax), which would mirror how end-users would work with the components, and the process would likely be less resource-intensive too.

TLDR; with Vite we could have one or more basic Demo pages to use as development "sandboxes", and then we could still use Storybook in an external repo for public documentation of the library. Thoughts?

However, doing this will make it more complicated to use Storybook as a tool during component development. It would still be useful for public documentation purposes (display all the merged components and how to use them, etc). Are folks relying on Storybook to help with the initial development of new components (similar to how we use unit tests)? Or do stories usually get written after the fact. @AnneT and @Catrope would be curious as to your thoughts here especially.

Yes, I definitely use Storybook a lot, both during initial development and when making changes to components.

Of course, this problem might be mitigated if we go with Vite, because that would give developers the ability to use a demo HTML page (which would also provide HMR during development). In some ways I could see this being easier to work with during the development process – you'd just write normal Vue code (instead of specialized Story syntax), which would mirror how end-users would work with the components, and the process would likely be less resource-intensive too.

That would work too, and as you say it wouldn't consume a gigabyte of memory while it's running.

TLDR; with Vite we could have one or more basic Demo pages to use as development "sandboxes", and then we could still use Storybook in an external repo for public documentation of the library. Thoughts?

Sounds good to me. But it might be nice to have the option of running a local Storybook, because some of the stories do exist for testing purposes, and I don't think we want to duplicate them all in demo pages. The basic story for every component also provides controls for tweaking all the props, which can be handy for testing while developing.

Sounds good to me. But it might be nice to have the option of running a local Storybook, because some of the stories do exist for testing purposes, and I don't think we want to duplicate them all in demo pages. The basic story for every component also provides controls for tweaking all the props, which can be handy for testing while developing.

These things may not be mutually exclusive – it may be possible to get both a Vite-powered Storybook page (using the latest version of Storybook with it's builder options) as well as a light-weight demo sandbox for when you want a quick scratch area without using so many resources. Will report back if/when I can get this all working.

@egardner Like Roan, I rely heavily on Storybook during component development. There's a lot of value in using a contributed tool with lots of existing features for this sort of thing (controls for interactive, configurable component demos, automatic documentation, accessibility checks, etc.) that many developers and designers already know how to use. That said, there are downsides: in addition to Storybook being resource-intensive and relying on webpack, designing stories for some components can be tricky (e.g. how to demonstrate the use of v-model when there's no easy or logical way to implement two-way binding for controls?) and documentation is limited (the generated code samples are not sufficient for our purposes, in my opinion). My instinct would be to try to make Storybook work for us, but I would be open to considering a custom solution if the benefits outweighed the costs. We would just need to closely evaluate the effort needed to build and maintain it.

Storybook will, from current point of view be an essential tool not only for devs, but also for designers, product managers and QTE folks.
The abilities it provides, being driven by a large developer community, is in my opinion far superior than any custom demoing solution.

Last time I checked storybook was pretty tightly integrated to webpack. I too find it useful during development.

Sorry if these are dumb questions, but I thought moving to Rollup was only for the built assets that we serve in production? Do we need to be concerned with an optional storybook install that uses webpack? Is storybook really a concern with switching to Rollup? My understanding of the outcome of the technical decision forum was that we were only concerned about the assets relating to a build step. Are there security issues with running webpack in CI that I've missed?

Yes! I'm currently wrestling with typescript (having some trouble getting Vite to emit type defs properly in WVUI) but then I'll try to migrate the storybook setup over.

Volker_E moved this task from Backlog to Doing on the WVUI board.

Closing as declined since we're deprecating WVUI for Codex (which uses Vite, built on top of rollup) shortly.

Change 694972 abandoned by Eric Gardner:

[wvui@master] [WIP] Bundle using Rollup

Reason:

https://gerrit.wikimedia.org/r/694972

Change 698012 abandoned by Eric Gardner:

[wvui@master] [WIP/Experimental] Build WVUI with Vite

Reason:

https://gerrit.wikimedia.org/r/698012