Page MenuHomePhabricator

Allow Parser::VERSION to be bumped without immediately resetting the ParserCache
Open, Needs TriagePublic

Description

Changes to Parser::VERSION cause all cached ParserOutput to be considered expired, leading to a massive load spike on high traffic wikis, potentially causing an outage or at least seriously degrading performance. This renders Parser::VERSION effectively useless.

To work around this, a configurable grace factor could be used: $wgParserVersionGraceFactor = 1 means consider all ParserOutput woth an old version expired right away. $wgParserVersionGraceFactor = 100 would mean only expire such outdated ParserOutput objects with a 1% chance.

Usage: before a Parser::VERSION bump is deployed, set $wgParserVersionGraceFactor = 100, and enable logging that would tell use how often a ParserOutput is discarded, how often a "bad" one is kept, and how often a "good" one is encountered. This would allows us to determine when $wgParserVersionGraceFactor can safely be set to a lower number (eventually, 1).

This mechanism could be limited to changes to the last (minor) segment of the VERSION string, with changes to other parts of the string always causing the ParserOutput to be discarded. Or, perhaps better, the grace factor should only applied to a specified previous value of Parser::VERSION, causing ParserOutput objects generated for an even older version to always be discarded even while the latest change is being ""trickled in". But considering how rarely the parser version changes, this is probably overkill.

Event Timeline

daniel created this task.Sep 7 2018, 11:05 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 7 2018, 11:05 AM
Anomie added a comment.Sep 7 2018, 2:13 PM

So a bunch of extra complexity to try to expire things slightly faster while still having breakage since 99% of the time it'll still use the outdated object?

daniel added a comment.Sep 7 2018, 2:19 PM

So a bunch of extra complexity to try to expire things slightly faster while still having breakage since 99% of the time it'll still use the outdated object?

Some complexity to expire things at a tunable rate, so we only use an outdated object 99% for a much shorter time than we do now.

But it would of course be better if we didn't need this. How about switching the ParserCache to JSON?

Reedy renamed this task from Allow Parser::VERSION to be bumped without immediately resetting hte ParserCache to Allow Parser::VERSION to be bumped without immediately resetting the ParserCache.Sep 7 2018, 3:21 PM
Tgr added a comment.Sep 7 2018, 5:09 PM

How about switching the ParserCache to JSON?

There's an approved RFC for that (T161647: RFC: Deprecate using php serialization inside MediaWiki), it just needs doing. Although if a code change adding a new private field is bad, a code change switching the serialization format... ugh.

I think a simple and useful step would be to have the MediaWiki version (branch name) of the generating code in the ParserOuput as a field. That makes adding hotfixes much simpler, and you can't forget bumping it.