Change Details

The ResourceLoader startup module specifies the current versions of all source code modules in production. It does so by iterating the registered modules and calling getVersionHash. For the VisualEditorDataModule class, this is implemented by producing the actual the script (getScript), and hashing its output. If I recall correctly, I originally introduced that approach as a way of not needing to keeping track of all possible ways in which interface messages can change. Especially because in VisualEditor in particular, there are **parsed** interface messages that allow templates. Which means tracking the message itself is usually not sufficient, it also needs to track when any of the templates used within are changed in anyway. When I first introduced this, the messages were relatively simple, but it seems at least on a few major wikis the messages have reached significant complexity that the overall startup time for ResourceLoader spends the majority of its run-time continuously parsing the same interface messages over and over again to see if it has changed. It has essentially changed the load.php?modules=startup request from a cheap check we do every 5 minutes, into a continuous uncached wikitext parser endpoint (like `action=parse`) that a lots (wikis * languages * skins) of concurrent polling loops going on. {F14757752} This currently has no parsing because in this particular context the Parser is called from the message interface, which doesn't have ParserOutput cache, due to the need to support message parameters and such. We need to figure out how to make this more performant. I suggest evaluating one of the following options: * See if we can make Message::parse() use ParserOutput in some way. This would essentially require the message to not have $N parameters, and not vary on things like PAGENAME. This should be feasible given we already don't support PAGENAME for these messages. The PAGENAME will always be "GlobalTitleFail". * If not, maybe MessageCache somehow. * If not, maybe unconditional APC cache based on page_touched. Template updates already propagate there afaik, it's just not being used right now.

The ResourceLoader startup module specifies the current versions of all source code modules in production. It does so by iterating the registered modules and calling getVersionHash. For the VisualEditorDataModule class, this is implemented by producing the actual the script (getScript), and hashing its output. If I recall correctly, I originally introduced that approach as a way of not needing to keeping track of all possible ways in which interface messages can change. Especially because in VisualEditor in particular, there are **parsed** interface messages that allow templates. Which means tracking the message itself is usually not sufficient, it also needs to track when any of the templates used within are changed in anyway. When I first introduced this, the messages were relatively simple, but it seems at least on a few major wikis the messages have reached significant complexity that the overall startup time for ResourceLoader spends the majority of its run-time continuously parsing the same interface messages over and over again to see if it has changed. It has essentially changed the load.php?modules=startup request from a cheap check we do every 5 minutes, into a continuous uncached wikitext parser endpoint (like `action=parse`) that a lots (wikis * languages * skins) of concurrent polling loops going on. {F14757752} This currently has no parsing because in this particular context the Parser is called from the message interface, which doesn't have ParserOutput cache, due to the need to support message parameters and such. We need to figure out how to make this more performant. I suggest evaluating one of the following options: * See if instead of Message::parse() we can use WikiPage/ParserOutput/ParserCache in some way. This would essentially require the message to not have `$N` parameters, and not vary on things like PAGENAME. This would give it the same performance as when viewing the interface message page through MediaWiki proper, with cascading updates from templates using `page_touched`, and the parser output cached nicely. Without additional infrastructure. We already don't support PAGENAME for ResourceLoader messages, so that should be fine. ** Downside: Cannot have `$N` parameters. (TODO: Is that a problem? Do we use them here?) ** Downside: Might have slightly different semantics between WikiPage::parse and Message::parse. should be feasible given we already don't support PAGENAME for these messages. The PAGENAME will always be "GlobalTitleFail". * See if we can wrap the Message::parse() call in a local cache of some kind (e.g. Memc or APC). ** Downside: Requires to figure out a way to purge the cache when needed, or requiring figuring out which case key would be safe enough to not need purging. Maybe we can use `page_touched`. That means we would still call Message::parse and preserve semantics that, but still have a cache that is similar to parser cache, but separate.

The ResourceLoader startup module specifies the current versions of all source code modules in production. It does so by iterating the registered modules and calling getVersionHash. For the VisualEditorDataModule class, this is implemented by producing the actual the script (getScript), and hashing its output. If I recall correctly, I originally introduced that approach as a way of not needing to keeping track of all possible ways in which interface messages can change. Especially because in VisualEditor in particular, there are **parsed** interface messages that allow templates. Which means tracking the message itself is usually not sufficient, it also needs to track when any of the templates used within are changed in anyway. When I first introduced this, the messages were relatively simple, but it seems at least on a few major wikis the messages have reached significant complexity that the overall startup time for ResourceLoader spends the majority of its run-time continuously parsing the same interface messages over and over again to see if it has changed. It has essentially changed the load.php?modules=startup request from a cheap check we do every 5 minutes, into a continuous uncached wikitext parser endpoint (like `action=parse`) that a lots (wikis * languages * skins) of concurrent polling loops going on. {F14757752} This currently has no parsing because in this particular context the Parser is called from the message interface, which doesn't have ParserOutput cache, due to the need to support message parameters and such. We need to figure out how to make this more performant. I suggest evaluating one of the following options: * See if we can makeinstead of Message::parse() we can use WikiPage/ParserOutput/ParserCache in some way. This would essentially require the message to not have $N`$N` parameters, and not vary on things like PAGENAME. This would give it the same performance as when viewing the interface message page through MediaWiki proper, with cascading updates from templates using `page_touched`, and the parser output cached nicely. Without additional infrastructure. This should be feasible given weWe already don't support PAGENAME for theseResourceLoader messages., The PAGENAME will alwaysso that should be "GlobalTitleFail"fine. * If not,** Downside: Cannot have `$N` parameters. (TODO: Is that a problem? maybe MessageCache somehow.Do we use them here?) * If not,* Downside: Might have slightly different semantics between WikiPage::parse and Message::parse. should be feasible given we already don't support PAGENAME for these messages. The PAGENAME will always be "GlobalTitleFail". * See if we can wrap the Message::parse() call in a local cache of some kind (e.g. Memc or APC). ** Downside: Requires to figure out a way to purge the cache when needed, or requiring figuring out which case key would be safe enough to not need purging. Maybe we can use `page_touched`. maybe unconditional APC cache based on page_touched.That means we would still call Message::parse and preserve semantics that, Template updates already propagate there afaikbut still have a cache that is similar to parser cache, it's just not being used right nowbut separate.