Page MenuHomePhabricator

Make FileModule version hash deterministic
Closed, ResolvedPublic

Description

As part of T98087, by default modules will use compute their version based on building the module and hashing its produced content.

However for the most common module, FileModule, I estimate this would slow down version computation by at least 10x. This would make If-None-Match/E-Tag handling of module responses too slow. And not to mention the startup module.

This is mainly due to involvement of preprocessors like Lessc that should only be invoked when necessary. While we can't prevent that in a generic way, we can have specific overrides for the FileModule.

Hashing only the files specified in the module is not sufficient because less-files may import other files. And CSS files reference images.

The logic for detecting changes is already in place from the timestamp-based system. Basically, when a module is build, it stores all files it had to access along the way in the database. Then, when computing a version, we check those files in addition to the files currently specified at the top level in the module definition.

It stands to reason that a module can only be different when one of the top level files changed, or when one of the previously known included files changed.

These references being tracked in the database is slightly suboptimal, but has been status quo for a long time and isn't an immediate concern beyond complexity (T90001).

Related Objects

Event Timeline

Krinkle created this task.Jul 7 2015, 6:55 AM
Krinkle updated the task description. (Show Details)
Krinkle raised the priority of this task from to Normal.
Krinkle claimed this task.
Krinkle added subscribers: ori, Krinkle, Aklapper, Catrope.
Krinkle added a comment.EditedJul 7 2015, 7:18 AM

I experimented with enabling module content versioning for all modules (except WikiModule, which can't retrieve content, not significant right now).

Locally with latest master, a handful of common extensions installed, warm localisation cache, and mainCache=anything (db). Using $ time curl -i ...load.php?modules=startup&only=scripts&debug=false and XhprofProfiler (StartProfiler: $wgProfiler, ProfilerXhprof, ProfilerOutputDb; view via w/profileinfo.php). Truncate mw.profiling table before the http request.

Before any change, real time is about 1.4s. Sorting profiling data by time%, from 65% and up:

NameTime (%)CountCalls/reqms/callkb/callms/req
main()100%11346.140346.14
ResourceLoader::respond85.23%11295.020295.02
ResourceLoaderModule::getVersionHash [+]73.89%2662660.960255.76
ResourceLoader::{array_ma closure} [+]69.89%4460.480241.91
ResourceLoader::getCombinedVersion [+]69.87%22120.930241.86
ResourceLoaderStartUpModule::getDefinitionSummary68.75%11237.970237.97
ResourceLoaderStartUpModule::getAllModuleHashes65.49%11226.680226.68
..
ResourceLoaderFileModule::getDefinitionSummary9.19%2332330.18042.3
ResourceLoaderFileModule::getFileMtimes7.16%2332330.14032.92
ResourceLoaderModule::safeFilemtime3.72%5875870.03017.12
DatabaseBase::select [+]4.74%25250.87021.79
DatabaseBase::query [+]3.54%26260.63016.31
MessageBlobStore::get [+]2.1%10100.9709.68
ResourceLoader::makeHash0.55%2692690.0102.54

After setting enableModuleContentVersion to true in ResourceLoaderModule and ResourceLoaderFileModule the real time from up from 1.4s to 8.5s.

NameTime (%)CountCalls/reqms/callkb/callms/req
main()100%113226.5803226.58
ResourceLoader::respond98.29%113171.5203171.52
ResourceLoaderModule::getModuleContent [+]97.82%221578.1203156.25
ResourceLoaderModule::buildContent [+]97.82%221578.103156.19
ResourceLoaderStartUpModule::getScript97.72%221576.4703152.94
ResourceLoaderStartUpModule::getModuleRegistrations96.79%113123.0203123.02
ResourceLoaderModule::getVersionHash95.41%26626611.5703078.59
ResourceLoaderFileModule::getStyles67.32%2362369.202172.01
..
ResourceLoaderFileModule::compileLessFile55.35%303059.5301785.8
lessc::compile [+]55.02%303059.1701775.24
DatabaseBase::select [+]12.6%6326320.640406.59
DatabaseBase::query [+]9.68%6326320.490312.39
MessageBlobStore::get [+]7.72%2622620.950249.08
ResourceLoaderFileModule::getScript [+]5.11%2362360.710167.19
ResourceLoaderFileModule::readScriptFiles4.82%2362360.670157.61

Main hot spots are less compilation (60%), MessageBlobStore (10%), and script/style file reading (20%).

MessageBlobStore still queries about the same number of rows, but is now fetching the blob field (to then sha1) instead of timestamp field. The preloader is currently not populating that so hence the bump there. I imagine without MessageBlobStore it would be much worse (as this store currently saves thousands of memcached and cdb queries, per T90001). Making the preload query include the blobs should help keep the DatabaseBase::select count around 25 instead of going up to 630.

I've also collected some numbers on overhead of computing file hashes vs mtimes.

From test.wikipedia.org using HHVM on mw1017:

PHPHHVM 3.6.1 (srv)
OSLinux
Inputresources/**/* (2241 files)
filemtime
warmups94.95210647583ms total on average (2 samples)
main81.240646044413ms total on average (30 samples)
sha1_file
warmups126.57392024994ms total on average (2 samples)
main124.19727643331ms total on average (30 samples)
md5_file
warmups87.63587474823ms total on average (2 samples)
main133.17994276683ms total on average (30 samples)

From localhost using PHP 5.6 on a MacBook Pro:

PHPPHP 5.6.10 (apache2handler)
OSDarwin
Inputresources/**/* (2242 files)
filemtime
warmups92.559576034546ms total on average (2 samples)
main82.975125312805ms total on average (30 samples)
sha1_file
warmups175.01509189606ms total on average (2 samples)
main175.9135723114ms total on average (30 samples)
md5_file
warmups173.68137836456ms total on average (2 samples)
main165.03329277039ms total on average (30 samples)

Source code of benchmark: P897.

While it hashing does almost twice as long, it's still less than 180ms for all 2000+ files in resources/ (images, less, css, js, i18n json, i18n js). An average module request would iterate over far fewer files. Even the startup module would be limited to a single skin/language combination.

Given these results I think we've got what we need to go ahead. Especially with the net benefit of improved cacheability (less trashing; T102578) this should work out well in the end.

Krinkle moved this task from Inbox to Doing on the Performance-Team board.Jul 9 2015, 11:35 AM

Change 223856 had a related patch set uploaded (by Krinkle):
resourceloader: Convert FileModule to use version hashing

https://gerrit.wikimedia.org/r/223856

Change 229471 had a related patch set uploaded (by Krinkle):
resourceloader: Convert FileModule to use version hashing

https://gerrit.wikimedia.org/r/229471

Change 229472 had a related patch set uploaded (by Krinkle):
resourceloader: Convert FileModule to use version hashing

https://gerrit.wikimedia.org/r/229472

Change 223856 merged by jenkins-bot:
resourceloader: Convert FileModule to use version hashing

https://gerrit.wikimedia.org/r/223856

Change 229471 merged by jenkins-bot:
resourceloader: Convert FileModule to use version hashing

https://gerrit.wikimedia.org/r/229471

Change 229472 merged by jenkins-bot:
resourceloader: Convert FileModule to use version hashing

https://gerrit.wikimedia.org/r/229472

ori closed this task as Resolved.Aug 5 2015, 9:30 PM