Page MenuHomePhabricator

Wikitext 2.0 as low-bandwidth transport for client-side rendering
Open, LowPublic

Description

This is a placeholder for a discussion we had at the 2018 parsing team off-site (suggested by @Sbailey) about using wikitext 2.0 as a low-bandwidth transport mechanism for client-side rendering.

The idea would be that wt2.0 would be highly optimized for render speed and bandwidth, while still looking as similar as possible to the wikitext we know and love. Instead of shipping <ul><li>Foo</li></ul> to the client, we ship *Foo\n (or {*Foo*}) and render to HTML on the client.

Another related thread to pull on: currently transclusions go into differently shaped "holes" in the resulting DOM tree (without getting into all of the details, think "block" and "inline" and trust me when I say there could be about a dozen other minor variants). In order to determine the right sort of hole to leave, you need to do a server query (roughly, fetch the TemplateData or equivalent for the transclusion and find what it says). In order to do fast incremental client-side rendering, the server might want to very-slightly annotate the wt2.0 it sends in order to indicate the type of each hole. Scarecrow: all transclusions could be normalized into {{#...}} forms (T204370) and then we'd use {{i#...}} to indicate an inline transclusion, {{b#...}} to indicate a block transclusion, and other single character annotations for other hole types. This would allow client-side rendering of the annotated wt2.0 stream without further server round-trips; it would then fill in each "hole" incrementally as the user scrolled down to make each visible with additional server requests for the wt2.0 to render for that hole.

Event Timeline

Note there will still need to be a way for clients without JavaScript enabled to view things.

Rather than calling this "wikitext 2.0", implying a replacement of the source language, it might make more sense to call it a new target for a parser to output rather than HTML.

ssastry subscribed.

This is an idea that surfaced at the office and not something we are working on right now, so prioritizing it Low for purposes of clarity.

This is very much WIP and not settled. I just wanted to document the conversations we had, since I think the bandwidth-reduction part is a genuinely new use-case/design criterion that we hadn't encountered before, and it could also motivate some of the "types for templates" ideas @ssastry has been working on.

If we ever get to a formal spec for wikitext, it might be structured in layers. There might be a core minimal layer meant for fast rendering and high bandwidth, with optional type annotations for faster template expansion (the "wikitext 2.0" discussed here); a layer above that might have user-friendly syntactic sugar (even forms like [[...]] might be considered syntactic sugar); and there might be another layer consisting of backwards-compatibility hacks to ease migration.

But "Wikitext 2.0" right now should really be thought of as just a catch-all term for "changes we think are interesting to discuss in the syntax/semantics/manipulation/rendering of wikitext".

And, to respond to @Anomie, there certainly would still need to be a way to do server-side rendering for non-JavaScript or search engine clients. That could be running JS server side like with the Marvin proposals, or running a PHP implementation of the "simplified wikitext renderer" (which again is hopefully very small and fast). Alternatively the code could be structured as "use Parsoid to render to DOM -> compress DOM to simplified wikitext -> render simplified wikitext", and then the non-JavaScript client would just get the Parsoid DOM directly bypassing the compress/render steps. Lots of options...