Right now we use a very basic but fast minifier. It has to perform very well due to the way we do on-demand package generation whilst having a very high cache hit ratio.
Though this is nice, it drastically limits our options and ability to implement additional features.
Three features in particular:
- Implementing source maps for easier debugging. At the moment with our basic minification enabling "Prettification" in Chrome Dev Tools makes the debugging experience "Okay" to deal with, but it is still all squashed into one file (doesn't map to original file names). When we do even more advanced minification this becomes even more important.
- Better minification: variable name changes, optimising for gzip, optimising statement to be shorter notation etc. 
So that's all great, but the problem is that, though UglifyJS (for example) is getting faster, it is still much too slow to run on many files at once on-demand from the web server.
Last February when I was in San Francisco, Roan and I have been thinking about something. I recall the following, though Roan might have a better version of this:
- We'd run the quick minifier on cache miss to populate the cache quickly and respond to the request. Then enqueue a job to run the advanced minifier (asynchronously).
- The job queue will then run the elaborate minification process and replace the cache item. We don't have to worry about the possibility of overwriting a new version with a new version because the cache keys contain a hash of the raw contents, so worst case scenario we're saving something that won't be used.
There's 2 details in particular I'm not sure about:
- How do we deliver them to the client? We have unique urls with version timestamps.
- The only way to trigger a purge is to either keep track of all urls in varnish that contain the module name and order a purge in varnish (after we update memcached, of course, so it'd be a quick roundtrip to Apache to compose a response from cached components)
- Or alternatively, cause a version bump in the module (touch() the files)
- The job queue, we can enqueue generic jobs that check everything. Or enqueue a job per cache item. In either case we need to account for the case that the enqueued job is no longer needed by the time it runs (in case we use generic jobs, once the first one runs, it should cancel any other ones in the queue, in case of module or item specific jobs cancel any for the same).
- 1: https://www.mediawiki.org/wiki/ResourceLoader/Features#On-demand_package_generation
- 2: http://www.html5rocks.com/en/tutorials/developertools/sourcemaps/ https://github.com/mozilla/source-map http://www.youtube.com/watch?v=HijZNR6kc9A
- 3: https://github.com/mishoo/UglifyJS2#compressor-options
- 4: https://github.com/mishoo/UglifyJS2