Page MenuHomePhabricator

Make resource files work offline (as through Service Workers) to improve performance and move toward full offline support
Closed, DuplicatePublic


I'd like to suggest implementing HTML5 caching manifests, so that supporting browsers could permanently cache resource files (HTML/CSS/JavaScript/image) used by Mediawiki.

Since implementing caching manifests would not be sufficient in itself to create an application to work wholly offline (such as outlined in bug 28706), this step would be for the sake of improving performance and lowering demand on the server, but could facilitate any future steps toward caching content as well.

After the first time obtaining copies of such resource files, browsers would never need to ask Wikipedia again for new copies, except for occasionally pinging Wikimedia servers (when online) to check whether there were any updates to those files that needed to be auto-downloaded.

Browsers of course currently do cache Wikipedia files, but they don't reserve permanent space in the browser for these files, so the browser will periodically need to re-download these files and thus slow down their visits and demand on the server.

Browsers may impose size limits on the amount of memory that can be used by a given domain for resource file caching, or require permission when beyond a specific size, so the user experience or availability may differ slightly based on the browser.

Documentation on manifests are here:

Version: 1.21.x
Severity: enhancement



Event Timeline

bzimport raised the priority of this task from to Low.Nov 22 2014, 1:36 AM
bzimport set Reference to bz45980.
bzimport added a subscriber: Unknown Object (MLST).

I don't think App Cache would play very well with ResourceLoader...

Why? Are the cache manifests not adequately dynamic? If this is based on experience, might you share your insights with WhatWG so the standard can be improved so as to accommodate your needs? It should I would imagine be possible to somehow and eventually work things out...

Is there a bug open for a hook to add attributes to the HTML tag? (I can't find one.) I mean, something like OutputPageHtmlAttributes, so that the manifest tag can be added?

Like I mentioned on the mailing list, ApplicationCache is meant specifically for single- or few-paged offline web applications. It will cache all pages permanently until the cache manifest is changed.

MediaWiki is *not* few-paged and is instead focused around separating content across individual pages. This is specifically addressed by the standard in section from the link in comment #0. Unfortunately, even there the exact implications of the prefer-online caching mode are not well documented.

Also, as Brion mentioned, this would definitely not play well with ResourceLoader, especially since it is already using two different layers of caching (the browser cache and the LocalStorage module cache). I'm assuming the ApplicationCache layer would have to fit in between those two layers.

Well, Application Cache can be used as part of making a "look up stuff" site like Wikipedia work offline. Though it does take an iframe hack and a bunch of wrestling:

ApplicationCache--assuming the difficulties of your integrating with ResourceLoader could be overcome--can still be of use:

  1. By utilizing a permanent cache, resource files don't need to be periodically redownloaded (e.g., if the browser auto-deletes its regular cache). It could optimize performance and minimize server requests (though browsers would periodically try pinging to see if the cache manifest file itself had been updated). In other words, the whole site doesn't need to be cached; appcache caches only those files designated, so appcache can be used even in environments not meant to go completely offline.
  2. In combination with my proposed functionality in bug 45981 , users would be able to selectively cache pages into a local IndexedDB database so that when offline, they could browse those cached files (and this appcache issue would be needed to ensure the application itself would still work in these cases).

Appcache doesn't only cache the designated files, it also implicitly caches the page that has the 'manifest' attribute. That page could be included in the NETWORK section of the manifest, so that it's always pulled from remote, but that would be tricky if the manifest is to be present on every wiki page.

I am currently using an application cache in an extension that makes SemanticForms available for offline data entry. To do this the manifest file is only loaded on particular pages, and lists all reasourceloader URLs along with the cache keys. It also lists all of the pages that need to be made available offline, and their last-modified dates.

Basically, it seems that the tricky part of appcache is making sure it's refreshed at the appropriate time.

matmarex set Security to None.
matmarex removed subscribers: Unknown Object (MLST), matmarex.

The HTML5 App Cache manifest is too limited to realistically be of any use to a complex web application like MediaWiki. Not for caching, and certainly not for an offline experience.

It was attempted in the past (c491891, 36983d11f) was eventually abandoned.

In addition, a better alternative is now emerging and the App Cache interface is considered deprecated or discouraged.

HTML Standard (Living Standard — Last Updated 1 October 2016)

7.9 Offline Web applications
This feature is being removed from the Web platform. [..] Using any of the features is highly discouraged. Use service workers instead.

Brettz9 renamed this task from Implement HTML5 caching manifests to Offlining resources.Oct 12 2016, 12:07 AM

I've changed the title to reflect the intent for any kind of support for offline resources. Service workers would also be in the spirit of my original suggestion.

Could you edit the task summary to describe an actual problem or a request for functionality, please?
"Offlining resources" does not tell me if this is a bug or a feature request and seems to miss a verb. Thanks!

Brettz9 renamed this task from Offlining resources to Offline resource files (as through Service Workers) to improve performance and move toward full offline support.Oct 12 2016, 11:14 PM
Brettz9 renamed this task from Offline resource files (as through Service Workers) to improve performance and move toward full offline support to Make resource files work offline (as through Service Workers) to improve performance and move toward full offline support.

To make MediaWiki page views work offline in a stable, maintainable and user-friendly manner, requires major refactoring to how our skinning system is applied.

I'm closing this in favour of the following task which is about establishing a practice of "page composition" (where the page would be logically composed from three pieces: wiki page content, the skin, and user information). That same proposal also includes a Service Worker to do the actual composition of these three pieces, and this Service Worker can run both server-side (in a Node.js service) and client-side (in modern browses).

T106099: RFC: Page composition using service workers and server-side JS fall-back