Page MenuHomePhabricator

Interwiki cache should be shared when Interwiki table is shared
Closed, ResolvedPublic

Description

Just a little poke for myself. There's a fair bit of redundancy with the fact that interwiki links are stored in the cache for each wiki that uses them, but that's all basically duplication when you are sharing the interwiki table.

Just a reminder to hunt through the code. I intend to make it so that when you share the interwiki table, the cache will use the key for that shared table instead of the local one. The data is all equivalent so the idea here is that doing that will reduce data duplication (Can get fairly large on wiki farms), and also improve the speed of the interwiki cache purging maintenance script since it will only need to run over a single database instead of once for each wiki.

Details

Reference
bz14164

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 21 2014, 10:12 PM
bzimport added a project: Wikimedia-Rdbms.
bzimport set Reference to bz14164.
bzimport added a subscriber: Unknown Object (MLST).

This should be doable in Title::getInterwikiCached();

wfForeignMemcKey and wfForeignWikiID will act as helpers for using foreign keys instead of locals.

Potentially a simple helper set wfMemcKeyForTable and wfWikiIDForTable could also be created, first input would be a table name and they would output the data for the shared if that table is shared, or local otherwise. That would simplify something which may likely end up common inside of MediaWiki when dealing with caches for strait table data.

unassign Daniel Friesen after talking to him

Krinkle set Security to None.
Krinkle removed a subscriber: wikibugs-l-list.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 23 2015, 10:21 PM
Krinkle updated the task description. (Show Details)
Krinkle added a subscriber: Krinkle.EditedMay 24 2017, 12:24 PM

While MediaWiki has generic support for sharing tables via wgSharedTables, not all tables are expected to be shared. The Interwiki table, for example, has no expectation of being shared. If it works for you, that's great, but any additional handling of it inside MediaWiki must account for the (more common) scenario of it varying by wiki. As such, the value in object cache for interwiki uses the regular makeKey() logic, which is prefixed by the current wiki's cache prefix. This could in theory be altered in ClassicInterwikiLookup by inspecting wgSharedTables and deciding to use makeGlobalKey() instead.

The current implementation is inefficient for large wiki farms, but for such users of MediaWiki, I would recommend not using a database table in the first place.

The underlying use case ("having a single source of interwiki configuration for all wikis with efficient caching"), is better accommodated by using a CDB file, or static PHP array. This is how Wikipedia sites are configured as well.

See:

Krinkle closed this task as Resolved.May 24 2017, 12:24 PM
Krinkle claimed this task.
Krinkle added a project: Performance-Team.