Page MenuHomePhabricator

Prototype a Vue SSR implementation using a Node service
Closed, ResolvedPublic

Description

Make a prototype of a server-side rendering implementation that uses a Node.js service to do the rendering.

High level architecture:

  • Make a Node.js service that accepts HTTP requests asking it to render a Vue component, and returns the rendered HTML (+ maybe some metadata)
  • When MediaWiki needs to render a Vue component, it gathers the necessary information (such as props to pass to the Vue component), makes an HTTP request to the service, then embeds the resulting HTML in its response
Current Status

A basic prototype of this extension lives here: https://gitlab.wikimedia.org/catrope/vuessrprovider. See the README file for details about installation and usage. The short version is:

So far we've had success server-rendering the basic Special:VueTest page which is added by the VueTest extension. With a little bit of additional work @Catrope was also able to server-render the Special:MediaSearch page from the MediaSearch extension (see this patch). Thus far none of our existing Vue code has been written with SSR in mind, so this prototype will probably uncover lots of things that will need to be changed upstream.

Future Tasks
  • TBD

Event Timeline

I built a prototype of how this approach might work. It's functional (though some things are still missing), and it helped me explore some of the architectural options and considerations.

To run the prototype yourself:

  • Download https://github.com/catrope/mw-vue-renderer, run npm install then npm start. This starts a service on port 8082.
    • If you're running MediaWiki using Vagrant, either run the service inside Vagrant (vagrant ssh, then cd /vagrant/path/to/mw-vue-renderer and npm start), or run the service outside Vagrant and set up reverse port forwarding with vagrant ssh -- -R 8082:localhost:8082
  • Check out https://gerrit.wikimedia.org/r/c/mediawiki/core/+/704650 in your MediaWiki install (note that the actual SSR implementation is in the parent commit, https://gerrit.wikimedia.org/r/c/mediawiki/core/+/699805 )
  • Go to Special:Blankpage. You should see a server-rendered demo component with some buttons and a bullet list
  • If you want to change the Vue code being rendered, edit resources/src/ssr-test/App.vue. You can also add additional files to the ssr-test ResourceLoader module, or create your own module and update the first parameter of the renderVueComponent() call in SpecialBlankpage.php accordingly.

Features:

  • Based on ResourceLoader modules: the only information the renderer needs is the name of the module that contains the Vue component, and the key in the module.exports object where it can find that component
  • require() works, both for other files in the same module (even virtual JSON files!) and for loading dependencies (including WVUI). This means you can have subcomponents and generally write code the way you normally would
  • i18n messages work the same way as they do in normal client code: they need to be listed in the messages property of the RL module and can be used through {{ $i18n( 'message-key' ) }} or <p v-i18n-html:message-key />
  • Data can be passed from PHP to the Vue component as props (and HTML attributes can be set too)
  • Hydration/mounting code is automatically generated and injected

Shortcomings / things that don't work yet:

  • Styles are loaded too late, so the server-rendered HTML initially loads without styling
  • Parsed i18n messages don't work. Adding mediawiki.jqueryMsg to the dependencies of your module crashes the rendering service (as does running any other code that tries to use mw globals or jQuery or the DOM, except for a few things that are mocked)
  • Page context config is not accessible, because mw.config is empty (calling mw.config.get( 'foo' ) doesn't crash, but always returns null during SSR)
  • The Icon component in WVUI doesn't work, because it tries to use the document global. We will have to make this component SSR-safe by moving browser-only code to lifecycle hooks that are not run in SSR mode (created runs both in the browser and in SSR, mounted in the browser but not in SSR)
  • Error handling is poor: if you try to do things that don't work (like use the icon component), you just get a blank page and a PHP notice. To see an error message that tells you what happened, you have to look at the console output of the mw-vue-renderer service.

Architecture overview:

mw-vue-renderer is a rendering service written in Node.js, using Express. It accepts HTTP POST requests to the /render URL. The POST body is a JSON blob containing modules, which contain files, which are strings of JavaScript or Vue code (the format used here is very similar to what ResourceLoader uses internally to send package modules to the client). The service parses and executes the code submitted to it, gets the component definition of the Vue component it needs to render, and renders it using Vue's SSR renderer. There's an separate server-side implementation of MediaWiki's i18n API, and the contents of i18n messages are sent to the service as part of the JSON blob.

In MediaWiki, there's a VueRenderer class that wraps around this service. It takes a single module name (the module containing the component to be rendered), and builds the JSON blob that the rendering service expects, with the contents of the module's files and its dependencies. In addition to passing back the rendered HTML it got from the service, it also generates a JavaScript snippet that can be used to mount and hydrate the HTML (i.e. make it interactive). This mount script is basically just a call to mw.hydrateVueSSR(), a new utility function that creates the same component as the rendering service did, and mounts it to the HTML generated by the rendering service. For convenience, OutputPage provides a renderVueComponent() method, which outputs the rendered HTML and adds the mount script. This makes rendering a Vue component on the server a matter of a single function call.

Note that the rendering service is stateless and doesn't know anything about MediaWiki or have access to its files. All the Vue code it executes (and all the i18n messages it references) is submitted to it over HTTP, and the same code is submitted again every time it needs to be rendered. This has the advantage of not having the service have to know how to access MediaWiki files or i18n messages, and it makes it easier to make advanced ResourceLoader features like virtual JSON files work. It is inefficient, but I'm hoping that that's not a major issue since the service is meant to run either on the same machine or in the same data center, so sending all that data over the network shouldn't be a problem. Building module contents on the MediaWiki side may be slow, and we may want to introduce some kind of caching (either on the Node service side or on the MW side) to address this.

Alternatives I considered:

  • Giving the rendering service access to MediaWiki's JS/Vue files, so that the HTTP POST data is just a module name and per-request parameters
    • Would be more difficult to deploy: the rendering service would either need to run on the same machine(s) as the PHP app servers, or the servers running the rendering service would need to be added as deployment targets
    • The rendering service would also have to obtain i18n messages somehow, possibly by duplicating some code from PHP (which involves complex caching and invalidation)
    • Would make it more difficult to support advanced RL features like virtual JSON files (but these are more of a nice-to-have)
  • Having the rendering service obtain the code and i18n it needs by sending HTTP requests to load.php
    • Would create a circular dependency graph between the rendering service and MediaWiki: MW sends an HTTP request to the service, which then sends an HTTP request to MW. There is precedent for this kind of architecture (Parsoid sending requests to api.php), but a lot of people disliked it and wanted it to be changed (Parsoid ended up removing this cycle as a side effect of porting to PHP).
    • Would incur more PHP new request overhead
    • Would support advanced RL features out of the box
Jdlrobson added subscribers: Unknown Object (User), Jdlrobson.Jul 23 2021, 6:40 PM

Styles are loaded too late, so the server-rendered HTML initially loads without styling

I assume this could be worked around by separating WVUI's style and script bundles into a separate styles module?

Parsed i18n messages don't work. Adding mediawiki.jqueryMsg to the dependencies of your module crashes the rendering service (as does running any other code that tries to use mw globals or jQuery or the DOM, except for a few things that are mocked)

I've been complaining about this for some time and previously tried to use T212521 as a driver to move us away from this. We need to refactor a lot of the code we have in core to avoid use of global variables.

It is posssible you can use the mock in @wikimedia/mw-node-qunit to get round this particular issue on the short term ?
https://github.com/wikimedia/mediawiki-extensions-QuickSurveys/blob/master/jest.setup.js#L3

Page context config is not accessible, because mw.config is empty (calling mw.config.get( 'foo' ) doesn't crash, but always returns null during SSR)

I guess config would need to be set directly, which is not a bad thing. I don't think components should be accessing mw.config directly but those should be passed in as props. It would be good if our eslinting code socialize us away from doing. At least inside Vector where we'd like to eventually using server side rendering I don't think this will be a problem. Likewise our messages could be passed in as props which isn't a terrible restriction.

Error handling is poor: if you try to do things that don't work (like use the icon component), you just get a blank page and a PHP notice. To see an error message that tells you what happened, you have to look at the console output of the mw-vue-renderer service.

That doesn't seem terrible. I imagine we'd want to show something a little friendly to users when this does happen rather than the full error.

Styles are loaded too late, so the server-rendered HTML initially loads without styling

I assume this could be worked around by separating WVUI's style and script bundles into a separate styles module?

It's more complicated than that. Styles don't just come from WVUI, they also come from end-user Vue components (the ones people write in MW core or in an extension/skin for the UI of a particular feature or special page). We could make them put styles into a separate module too, but that breaks the single-file component concept in Vue. The Structured Data team has already complained about this issue (they had to move styles out of their .vue files to make them render-blocking), so I've already been thinking about ways to handle this. Ideally I would like to be able to load the module's styles first (as render-blocking styles), and then later only load the scripts+messages etc without the styles, but that would require changes to ResourceLoader and is a departure from how we've previously addressed this issue (and for that reason, RL actively fights this; calling addModuleStyles() on a module that isn't styles-only silently fails). Alternatively, we could make a separate styles module that looks at the same .vue files and only extracts the styles, but I think that would result in ugly module definitions. So I'd prefer to purse the former approach (and make changes to RL) in a way that avoids double-loading and provides ways to reduce the amount of CSS that is loaded as render-blocking to only the styles that are needed (maybe through a LESS mixin or construct that marks which styles should be render-blocking and which shouldn't).

cc @Krinkle who I'm sure would be interested in the above.

Parsed i18n messages don't work. Adding mediawiki.jqueryMsg to the dependencies of your module crashes the rendering service (as does running any other code that tries to use mw globals or jQuery or the DOM, except for a few things that are mocked)

I've been complaining about this for some time and previously tried to use T212521 as a driver to move us away from this. We need to refactor a lot of the code we have in core to avoid use of global variables.

It is posssible you can use the mock in @wikimedia/mw-node-qunit to get round this particular issue on the short term ?
https://github.com/wikimedia/mediawiki-extensions-QuickSurveys/blob/master/jest.setup.js#L3

That would be a good place to start. I was also going to look at using banana-i18n to support plurals etc.

Page context config is not accessible, because mw.config is empty (calling mw.config.get( 'foo' ) doesn't crash, but always returns null during SSR)

I guess config would need to be set directly, which is not a bad thing. I don't think components should be accessing mw.config directly but those should be passed in as props. It would be good if our eslinting code socialize us away from doing. At least inside Vector where we'd like to eventually using server side rendering I don't think this will be a problem.

I think that's a good idea: discourage use of mw.config and encourage passing in per-request config data as props.

Likewise our messages could be passed in as props which isn't a terrible restriction.

I don't think that would work as well, because it would make the code much more annoying to write, and it doesn't work for parameterized messages whose parameters change dynamically on the client (e.g. "you have N new notifications" needs a client-side i18n system with plural support to rerender the message when N changes).

Error handling is poor: if you try to do things that don't work (like use the icon component), you just get a blank page and a PHP notice. To see an error message that tells you what happened, you have to look at the console output of the mw-vue-renderer service.

That doesn't seem terrible. I imagine we'd want to show something a little friendly to users when this does happen rather than the full error.

Yes, and to developers too :) . But this is just a prototype, so I didn't spend much time on that.

I don't think that would work as well, because it would make the code much more annoying to write, and it doesn't work for parameterized messages whose parameters change dynamically on the client (e.g. "you have N new notifications" needs a client-side i18n system with plural support to rerender the message when N changes).

A little annoying, but sure. I'm willing to deal with a little annoyance for server side rendered components.

In case it's useful, in SkinMustache we got around this by defining messages up front
https://github.com/wikimedia/Vector/blob/master/skin.json#L33

and adding them to the template properties:
https://github.com/wikimedia/mediawiki/blob/master/includes/skins/SkinMustache.php#L196

but explicitly disallowing parsed messages.

When MediaWiki needs to render a Vue component, it gathers the necessary information (such as props to pass to the Vue component), makes an HTTP request to the service, then embeds the resulting HTML in its response

Are you thinking this would be a just-in-time rendering step, executed as part of the MW request-handling process?

If so, I assume that much (though maybe not all) of the missing information would already be available to the server before the request to the rendering service gets fired – message strings, user status, etc. In that case, perhaps the request context (or a subset of it) could be included as parameters?

To phrase this another way, what if there was a way in PHP to explicitly add parameters when the request to the external service is made, so that specific bindings can be made visible to the Vue component's SSR context? Kind of like how we do when rendering Mustache templates (but with an added HTTP request in the middle).

When MediaWiki needs to render a Vue component, it gathers the necessary information (such as props to pass to the Vue component), makes an HTTP request to the service, then embeds the resulting HTML in its response

Are you thinking this would be a just-in-time rendering step, executed as part of the MW request-handling process?

Yes.

If so, I assume that much (though maybe not all) of the missing information would already be available to the server before the request to the rendering service gets fired – message strings, user status, etc. In that case, perhaps the request context (or a subset of it) could be included as parameters?

To phrase this another way, what if there was a way in PHP to explicitly add parameters when the request to the external service is made, so that specific bindings can be made visible to the Vue component's SSR context? Kind of like how we do when rendering Mustache templates (but with an added HTTP request in the middle).

Yes, we could do that. Instead of requiring each SSR caller to manually obtain and pass in their own server-side information, we could make some of it available everywhere. However, I think we should be careful with that, because we should remember that this code also needs to be runnable on the client. That means that the same information needs to be available in the same place in the browser. That will work for some things like mw.user.getName() or mw.config.get( 'wgPageName' ), but we shouldn't add too many new things, to avoid sending information to the client on every page that may not be needed. I'm also hesitant to have SSR code rely too much on mw.user and mw.config, because it's going to be confusing if only a limited number of things are available in SSR mode. But basic request context stuff probably should be made available, yes.

As for i18n message strings, those are already sent to the service, based on which message keys are declared in the ResourceLoader module definition of the module being rendered.

@egardner suggested putting the SSR implementation in a MediaWiki extension, with a basic fallback in MediaWiki core that renders an empty div if SSR is not available. I've redone my prototype to follow that architecture at https://gitlab.wikimedia.org/catrope/vuessrprovider . It's missing a few i18n features from the previous prototype, but it's also a bit cleaner.

Did you guys heard about Remix? https://remix.run/

It can probably help on optimising rendering process.
I see it has been tested with Vue.js too:
https://www.smashingmagazine.com/2022/07/look-remix-differences-next/

Did you guys heard about Remix? https://remix.run/

It can probably help on optimising rendering process.
I see it has been tested with Vue.js too:
https://www.smashingmagazine.com/2022/07/look-remix-differences-next/

Thanks for the link, I'd heard of Remix but thought it was only for React/Next.js. Bun also looks very interesting, though it's still very new. These new "SSR-focused" runtimes may help a lot in terms of performance optimization.

For now we are prototyping using a standard Node.js runtime, but we should keep an eye on developments here.

We have a working prototype now, so this task can be closed. Further SSR work can be tracked separately.