Page MenuHomePhabricator

Implement composition service library
Closed, DuplicatePublic


This task is about implementing a minimal service that essentially composes an HTTP response based on multiple other resources that are themselves more cacheable and less variable. Initial version requirements:

  • High-throughput. Suitable for handling traffic at the edge, essentially doing only HTTP and string manipulation.
  • Request routing.
  • HTML Templating. Fetch, precompile and cache templates - presumably Mustache.
  • Streamable. Must implement template handling so flushing starts early and continues progressively.
  • Export as ServiceWorker. Add an endpoint that exports a JavaScript program compatible with a ServiceWorker that contains all the utilities (elematch, mustache etc.), Router, RequestHandler, and the current install's router configuration and custom RequestHandlers.


It should probably use service-runner and not be specific to MediaWiki in anyway. We can later use this library in a MediaWiki-specific installation that extends certain base classes and provides relevant configuration.

Due to unsolved technical debt (see T111588#2416137), I suggest the initial implementation is used for a less complicated use case. The portal, for example, which needs localisation in a performant way. Currently done client-side with XHR and causes a FOUC.

Draft spec by @GWicke and @Krinkle:

  • node-serviceworker-server: Node library that runs an HTTP service. It can be configured to map a domain and request scope to a service worker url. The service will use the node-serviceworker library to turn the service worker script into something that we can instantiate and make a request to on the server-side.
    • Status: Concept
  • node-serviceworker: Node library that provides a Node sandbox with several browser APIs available in its scope (such as Fetch, ServiceWorker Cache, and more).
  • sw-wikimedia-helpers: Collection of utilities we expect most of our ServiceWorker clients to need. Such as request routing, view abstraction, and a command-line script to generate a compact sw.js file. This utility library will likely make use of:
    • browserify
    • mixmaster: Produce a readable stream from an array of string literals, functions, promises, and other streams. With the option to pass through one or more transforms. This allows progressively streaming to the client with the ability to dynamically substitute portions, and to precompile any templates.
    • elematch: Efficient matching of elements in a stream of HTML. To be used with Mixmaster. This would allow to progressively stream to the client with the ability to dynamically substitute portions.
    • musti: Streamable Mustache renderer. Uses Mixmaster.

Event Timeline

Krinkle created this task.Jul 18 2016, 5:15 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptJul 18 2016, 5:15 PM
Krinkle moved this task from Inbox to Radar on the Performance-Team board.Jul 18 2016, 6:54 PM
Krinkle triaged this task as Medium priority.Jul 18 2016, 6:57 PM
Krinkle updated the task description. (Show Details)Jul 18 2016, 9:53 PM
Krinkle renamed this task from Implement minimal composition service to Implement composition service library.Jul 18 2016, 10:13 PM
Krinkle updated the task description. (Show Details)
Krinkle updated the task description. (Show Details)Jul 18 2016, 10:16 PM
GWicke added a comment.EditedJul 28 2016, 12:41 AM

Quick status update:

Agabi10 added a subscriber: Agabi10.Aug 3 2016, 8:42 PM
Krinkle updated the task description. (Show Details)Aug 3 2016, 9:04 PM
Scott_WUaS updated the task description. (Show Details)Aug 3 2016, 10:10 PM
Scott_WUaS added a subscriber: Scott_WUaS.