Page MenuHomePhabricator

Performance regression in wmf15 roll-out
Closed, ResolvedPublic

Description

The roll-out of wmf15 to non-Wikipedias appears to have introduced a significant performance regression, which registered as a 1-second increase in median page load times across the cluster. The regression went away when I reverted https://gerrit.wikimedia.org/r/#/c/188409/.

Changelog: https://www.mediawiki.org/wiki/MediaWiki_1.25/wmf15

Event Timeline

ori raised the priority of this task from to Unbreak Now!.
ori updated the task description. (Show Details)
ori added subscribers: ori, greg.

Looks like the rollback messed up ResourceLoader caching somehow, see T88576.

Group1 was put back on 1.25wmf15 at 2015-02-04T20:33Z. mediaWikiLoadComplete.desktop timings rose again, but the rise started a ~19:40Z and has basically held steady since.

If all is well over night, the plan is for @Reedy to push out wmf16 to testwikis (group0) and wmf15 to the wikipedias (group1 and group2).

greg claimed this task.
20:30 logmsgbot: reedy rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.25wmf16
20:29 logmsgbot: reedy rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedias to 1.25wmf15

Back on track.

I think we look good enough, given everything that's happened today/yesterday with outages and loss of graphite data: