Page MenuHomePhabricator

MessageCache does not work due to WAN cache stub
Closed, ResolvedPublic

Description

Mediawiki 1.31.1
phpVersion 7.0.33-1~dotdeb+8.1
$wgMessageCacheType = CACHE_ACCEL; (APCu)
$wgCacheDirectory = '/tmp/mw-cache-wikiname';
database load via $wgLBFactoryConf.

The debug error message:

[MessageCache] MessageCache::load: Loading zh-hans... local cache has the wrong hash, global cache is presumed expired, loading from database
[caches] parser: RedisBagOStuff
Article::view using parser cache: yes
[MessageCache] MessageCache::load: Loading zh... local cache has the wrong hash, global cache is presumed expired, loading from database
[MessageCache] MessageCache::load: Loading en... local cache has the wrong hash, global cache is presumed expired, loading from database
[MessageCache] MessageCache::load: Loading zh-hant... local cache has the wrong hash, global cache is presumed expired, loading from database
[MessageCache] MessageCache::load: Loading zh-cn... local cache has the wrong hash, global cache is presumed expired, loading from database
[MessageCache] MessageCache::load: Loading zh-tw... local cache has the wrong hash, global cache is presumed expired, loading from database

For some reason, the message cache never function and keep loading from database.
We do find many MessageCache in APCu, and CDB files like l10n_cache-en.cdb in /mw-cache-wikiname (with proper owner www, and 755 folder rights, 644 file rights).
Changing $wgMessageCacheType from CACHE_ACCEL to 'redis' does not solve this problem.

Is this a Mediawiki bug, or we are not caching message in the right way?

Thank you for your help in advance!

Event Timeline

Zoglun created this task.Feb 10 2019, 2:53 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 10 2019, 2:53 PM
alex-mashin added a subscriber: alex-mashin.EditedJul 6 2019, 7:15 AM

It seems that I have a similar problem, too (MW 1.31.1, PHP 7.2).

Queries like the one below are run every several seconds:

SELECT /* MessageCache::loadFromDB(en)-small  */  page_title,old_id,old_text,old_flags  FROM `page` JOIN `revision` ON ((page_latest=rev_id)) JOIN `text` ON ((rev_text_id=old_id))   WHERE page_is_redirect = '0' AND page_namespace = '8' AND (page_title LIKE '%/en' ESCAPE '`' ) AND (page_len <= 131072);

Cache configuration:

$wgMessageCacheType = CACHE_ACCEL;
$wgUseLocalMessageCache = true;
$wgMaxMsgCacheEntrySize = 131072;
$wgCacheDirectory  = '/var/cache/mediawiki';
$wgLocalisationCacheConf = [
	'class'			=> LocalisationCache::class,
	'store'			=> 'array',	// Use opcache.
	'storeClass'		=> false,	//
	'storeDirectory'	=> false,	// == $wgCacheDirectory
	'manualRecache'		=> true,	// -- no update on web requests. Use rebuildLocalisationCache.php.
];
$wgMsgCacheExpiry = 2 * 24 * 60 * 60;	// -- two days.

.php files in /var/cache/mediawiki exist and are readable and writable by www-data.

Opcache is up and running, opcache.file_cache_only = 0.

The following debug information is in page source code:

[caches] cluster: APCBagOStuff, WAN: mediawiki-main-default, stash: db-replicated, message: APCBagOStuff, session: SqlBagOStuff
[caches] LocalisationCache: using store LCStoreStaticArray

maintenance/rebuildLocalisationCache.php would not rebuid localisation caches without --force, thinking that they are still fresh.

alex-mashin renamed this task from MessageCache failed with/without $wgMessageCacheType+$wgCacheDirectory to MessageCache does not work due to empty hash.Jul 10 2019, 1:42 AM
alex-mashin triaged this task as High priority.
alex-mashin added a comment.EditedJul 14 2019, 4:26 PM

I spent my whole weekend to find a workaround for this bug, tearing through undebuggable spaghetty callback hell.

MediaWiki tries to get messages from EmptyBagOStuff, of course, finding nothing. This, in turn, happens because MW tries to use WAN cache, which is irrelevant for my single-server instance of MediaWiki, for I don't own a datacenter, which I am sure is the case for the vast majority of MediaWiki users.

To stop my database from beeing flooded with queries for messages on any web request after getting nothing from EmptyBagOStuff wrapped by WAN cache object, I had to add this stub to my LocalSettings.php, creating a mock WAN cache—in fact, CACHE_ACCEL:

// See https://phabricator.wikimedia.org/T215726:
$wgWANObjectCaches [$wgMainCacheType] = [
    'class'     => WANObjectCache::class,
    'cacheId'   => $wgMainCacheType,
    'channels'  => []
];
$wgMainWANCache = $wgMainCacheType;
alex-mashin renamed this task from MessageCache does not work due to empty hash to MessageCache does not work due to WAN cache stub.Jul 15 2019, 5:02 AM

WOW @alex-mashin really thank you for your time. I will try your fix in my environment.

Krinkle updated the task description. (Show Details)
Krinkle updated the task description. (Show Details)
Krinkle updated the task description. (Show Details)
Krinkle added a subscriber: Krinkle.EditedJul 15 2019, 10:52 PM

@alex-mashin The WANObjectCache feature is the default interface for almost all caching in MediaWiki. If you disable this, your wiki will be slow.

Whether you have a data centre or not, does not matter :-) It is named only this way because it can support coordination between multiple servers and data centres.

By default in MediaWiki, and even for Wikipedia, WANObjectCache is configured to use wgMainCacheType as its cache backend. There should be no need to change this configuration because this is already the default.

However, it is important for any MediaWiki installation to set wgMainCacheType to something other than CACHE_NONE.

If you have a single-server set up, and do not yet need Redis or Memcached to store more values at the same time, then CACHE_ACCEL is indeed a good choice for wgMainCacheType, and WANObjectCache (which is where most caching happens) will then automatically use it.

See also https://www.mediawiki.org/wiki/Manual:Performance_tuning.

Krinkle closed this task as Resolved.Jul 15 2019, 10:53 PM
Krinkle claimed this task.
alex-mashin added a comment.EditedJul 16 2019, 1:09 AM

@Krinkle, my $wgMainCacheType was set to CACHE_ACCEL. If the task author's wasn't, his $wgMessageCacheType was, and was effectively ignored.

The problem is the fact that the default $wgWANObjectCaches does not have a corresponding entry, and the fallback is the database.

At best, the problem is with the default value of $wgWANObjectcaches or with how it is documented.

This is the default WANObjectCache configuration setup Setup.php. This code runs after the loading of your LocalSettings.php file:

Setup.php
	$wgMainWANCache = 'mediawiki-main-default';
	$wgWANObjectCaches[$wgMainWANCache] = [
		'class'    => WANObjectCache::class,
		'cacheId'  => $wgMainCacheType
	];

This creates a the first (and usually, only) key of wgWANObjectCaches, named 'mediawiki-main-default' and it uses the backend configured by your LocalSettings.php file for $wgMainCacheType.

This means that anything you configure as $wgMainCacheType is automatically used by WANObjectCache.

If this is not happening for you, then there might be something else in your LocalSettings.php file or an extension causing that to break.