Page MenuHomePhabricator

Cache update time for Gadgets and Common.js/css regressed from 5 min to 1 hour
Closed, ResolvedPublic

Description

In ruwiki in the recent days, after you save a gadget page, it takes too long for the change to take effect on pages. Usually it took about 5 minutes. Now it takes an undefined amount of time in a range of 15 minutes to 1 hour.

Event Timeline

Restricted Application added a project: Performance-Team. · View Herald TranscriptJul 12 2018, 8:35 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Mvolz added a subscriber: Mvolz.Jul 12 2018, 9:19 AM

Have you tried purging the page manually as a quick fix for this? https://www.mediawiki.org/wiki/Manual:Purge

Sure, purge is useless in such cases.

stjn awarded a token.Jul 12 2018, 3:36 PM
SerDIDG added a subscriber: SerDIDG.
Imarlier assigned this task to Krinkle.Jul 17 2018, 8:08 PM
Imarlier moved this task from Inbox to Next In This Quarter on the Performance-Team board.
Krinkle triaged this task as High priority.Jul 19 2018, 3:52 AM

I've confirmed the bug on ru.wikipedia.org when making a change to <https://ru.wikipedia.org/wiki/MediaWiki:Gadget-markadmins.js. Before the edit, the version was: ["ext.gadget.markadmins","1j79x0v",[] (for modules=startup&lang=ru), and even when bypassing Varnish cache or waiting 5min, it remained at the same version.

I'm unable to reproduce this on test2.wikipedia.org with just adding or removing a gadget. Both for removal and addition, Special:Gadgets and Preferences are updated immediately, and within 5min the startup registry is also updated.

I'll try again to reproduce on test2wiki later with an edit (not just add/removal of entire gadget), and also to rule out if it has to do with lang=ru on ruwiki, or also reproducible with lang=en on ruwiki and/or lang=ru on test2wiki etc.

I suspect this may have to do with the titleInfo cache and the on-edit callback to purge it not working for some reason.

Nirmos added a subscriber: Nirmos.Jul 23 2018, 5:53 PM

It has nothing to do with lang=ru. It's the same problem on svwiki.

I do not know if this can help, and if it is relevant at all...
In the last month changes in common.css work too slow. When adding a new style value for all pages, it takes at least 15 minutes to see them. Ctrl-F5 and purge do not help.

MBH added a subscriber: MBH.Jul 23 2018, 7:26 PM

I can reproduce this on test2.wikipedia.org by editing MediaWiki:Gadget-modrollback.js and confirming that, even bypassing Varnish through extraneous query parameters and/or XWD, it doesn't update within the expected 5 minutes.

I suspect this is caused by an MCR patch.

ResourceLoaderWikiModule::preloadTitleInfo calls WANObjectCache::getWithSetCallback with the following checkKey:

// $cache = ObjectCache::getMainWANInstance();
// $db = wfGetDB( DB_REPLICA );
$cache->makeGlobalKey( 'resourceloader', 'titleinfo', $db->getDomainID() )

ResourceLoaderWikiModule::invalidateModuleCache is called from three places, as needed:

  1. WikiPage::doDeleteUpdates - this call looks fine, except that it uses wfWikiId() instead of Database::getDomainId(), but at least those are identical at the moment. I've confirmed this code path to work.
  2. DerivedPageDataUpdater - this call looks broken. It calls getWikiId which unconditionally returns false with a TODO comment. Oops?
  3. Title::invalidateCache - this call seems fine, although it is suspicious that it doesn't seem to get called from WikiPage::onArticleEdit and not from DerivedPageDataUpdater either. Perhaps an unrelated regression.
Krinkle renamed this task from Gadgets update time has increased drastically to Cache update time for Gadgets and Common.js/css regressed from 5 min to 1 hour.Jul 25 2018, 9:58 PM
Krinkle removed a project: Gadgets.

Moreover, it's true for user JS/CSS as well. As I remember, those used to be updated instantly.

I confirm about user css. I've waited around half an hour while cache dropped.

daniel added a subscriber: daniel.Aug 3 2018, 12:46 PM
  1. DerivedPageDataUpdater - this call looks broken. It calls getWikiId which unconditionally returns false with a TODO comment. Oops?

getWikiId() returning false is correct, as throughout core, "false" is used to mean "local wiki" in $wikiId parameters.
However, invalidateModuleCache() does *not* support this. I missed that, and you are right that this is likely the problem here. I'll make a patch.

  1. Title::invalidateCache - this call seems fine, although it is suspicious that it doesn't seem to get called from WikiPage::onArticleEdit and not from DerivedPageDataUpdater either. Perhaps an unrelated regression.

WikiPage::insertOn() bumps page_touched directly, so Title::invalidateCache is not needed. PageUpdater uses Title::invalidateCache for null-edits. So that seems fine.

Change 450217 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/core@master] Don't pass false as a wikiId to invalidateModuleCache()

https://gerrit.wikimedia.org/r/450217

  1. DerivedPageDataUpdater - this call looks broken. It calls getWikiId which unconditionally returns false with a TODO comment. Oops?

getWikiId() returning false is correct, as throughout core, "false" is used to mean "local wiki" in $wikiId parameters.

Thanks clarifying. I assumed (wrong) that this method getWikiId related to the "wiki id" as used in context of wfWikiId, SiteConfiguration, cache keys, and cookie prefixes; we don't support false there.

But I understand now, the method relates to the parameter for LBFactory and Database methods that accept a foreign dbname or false (false being current local wiki). To avoid exactly this confusion, and in preparation for the classes becoming their own re-usable library, Aaron renamed this parameter a while back from $wiki to $domain (and getWikiId to getDomainId).

Change 450217 merged by jenkins-bot:
[mediawiki/core@master] Don't pass false as a wikiId to invalidateModuleCache()

https://gerrit.wikimedia.org/r/450217

Change 450294 had a related patch set uploaded (by Krinkle; owner: Daniel Kinzler):
[mediawiki/core@wmf/1.32.0-wmf.15] Don't pass false as a wikiId to invalidateModuleCache()

https://gerrit.wikimedia.org/r/450294

Change 450294 merged by jenkins-bot:
[mediawiki/core@wmf/1.32.0-wmf.15] Don't pass false as a wikiId to invalidateModuleCache()

https://gerrit.wikimedia.org/r/450294

Krinkle closed this task as Resolved.Aug 4 2018, 4:40 PM

Tested and verified on test2.wikipedia and ru.wikipedia.

MBH removed a subscriber: MBH.Jan 20 2019, 2:53 PM