Some general notes:

• No cache that I found is explicitly split between Repo and Client functionality. Wikibase has shared cache key configuration settings, with a sharedCacheKeyGroup that’s more general and a sharedCacheKeyPrefix that’s more specific; in production, the sharedCacheKeyGroup is wikidatawiki or testwikidatawiki, and the sharedCacheKeyPrefix is the group plus the current train version. Some caches explicitly use one of those settings as part of their key to share the cache between Repo and Client; one cache uses the same string, containing “repo”, in both Repo and Client; others don’t put anything special in the cache key.
• Whether the cache is shared between different wikis depends on how the key is created. makeKey() creates a key local to the current wiki, whereas makeGlobalKey() creates one that is shared between all wikis. Wikibase also has a SimpleCacheWithBagOStuff class, which always uses makeKey(), though I’m not sure whether some of those caches wouldn’t benefit from a global key instead. Some caches don’t use anything special to create the key, which is probably bad.

Breakdown of the caches:

Misc notes:

• ClientHooks::getSiteConfiguration() passes WanObjectCache options into a BagOStuff, which is probably a bug, but hopefully a harmless one.
• CachingPropertyInfoLookup uses two cache layers, one for the whole cluster and one for the current server.
• CachingPropertyInfoLookup works together with CacheAwarePropertyInfoStore, which takes care to update the cache when the property info changes. That said, I’m not sure if CacheAwarePropertyInfoStore correctly updates both cache layers that CachingPropertyInfoLookup uses? To me it looks like it might only updates the cluster layer, not the server layer.
• The property info data cached by those two classes ultimately comes from JSON blobs controlled by Repo (wb_property_info table, pi_info column), so Client train versions must be able to deal with data of the Repo train version regardless of how stale or fresh the cached data is.
• SqlEntityInfoBuilder is slated for removal (T254283), as is DatabaseEntityInfoBuilder, whose caching bits look suspiciously similar anyways.

IMO adding almost all of the info in the above comment to a topic under https://doc.wikimedia.org/Wikibase/master/php/md_docs_topics_index.html would be super useful!

Thank you for doing this investigation, it's pretty well thought and extensive. One thing that would be useful to have is to know the transitive usage of the caches. For example it says it's used in lib but that code might have only been used in client and not repo.

A few other thoughts from my side (mostly tickets we should create and add to the backlog)

• AbstractTermPropertyLabelResolver should start passing the key into makeKeyGlobal (TODO ticket)
• EntityRevisionCache with its current usage should probably be going away once we move to use more modern RevisionStore methods. T255305: [Investigation] - Consider if the Wikibase cache for CachingEntityRevisionLookup is needed any more
  • SqlBlobStore which RevisionStore uses already caches content in Memcached as needed
  • Having said that, it is currently also currently using in client with a HashBagOStuff
• Nonetheless, EntityRevisionCache should also use makeKeyGlobal (TODO ticket)
• SimpleCacheWithBagOStuff should allow for instance creations with differently scoped keys (TODO ticket)
  • Either always global, and let users of the cache determine if the key needs to be namespaces in some way
  • Or allow construction with some value to indicate global or local
• Looking at a very surface level to the uses of SimpleCacheWithBagOStuff currently, it looks as if all uses of the cache were intended to use a shared global cache.
  • We added the makeKey call around 3 months ago https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Wikibase/+/583199/ which altered this (before they were global)
  • super interesting that the cache hit rate is still 95-98% for most of the caches there even when split per wiki.
  • Of course, using a global cache introduces the need to keep compatibility between versions, however, if the data being stored changed between versions the cache key should be updated to avoid issues.
• CachingPropertyOrderProvider is configurable per wikibase site, so that cache probably should be split per wiki, even if for wikidata.org using a single key might make sense.

One thing that would be useful to have is to know the transitive usage of the caches. For example it says it's used in lib but that code might have only been used in client and not repo.

Two components are marked as being defined in Lib but only used in Client; as far as I can tell, all the other Lib components are used in both Repo and Client.

CachingPropertyOrderProvider is configurable per wikibase site, so that cache probably should be split per wiki, even if for wikidata.org using a single key might make sense.

I was surprised to learn that this functionality is relevant to the client at all, but it’s exposed through Lua (and mwgrep reports a fair amount of usages of orderProperties, mostly in copies of Module:Databox and Module:Universal infocard). I’m also surprised that it apparently checks for the MediaWiki:Wikibase-SortedProperties message on the client wiki first before falling back to the repo wiki, so I checked if that message actually exists on clients – according to P11588, Wikidata and Commons are the only wikis that have that page.

IMO adding almost all of the info in the above comment to a topic under https://doc.wikimedia.org/Wikibase/master/php/md_docs_topics_index.html would be super useful!

I’m worried that parts of it, specifically the contents of the table, will get outdated fairly quickly… I hope that there are some code changes we can make that would make it easier to re-collect the information in that table, and then the documentation could be limited to more general comments (e. g. difference between makeKey and makeGlobalKey), maybe?

As the perfect example:

• DatabaseEntityInfoBuilder
• SqlEntityInfoBuilder

can already disappear

I don't have anything in particular to add, apart from a thank you to @Lucas_Werkmeister_WMDE for athering this info in one table (the overview of cache components is really helpful for a noob like me to learn more).

I created three tasks for the TODOs Adam mentioned in T255116#6233460:

• T255982: Use makeGlobalKey in AbstractTermPropertyLabelResolver
• T255983: Use makeGlobalKey in EntityRevisionCache
• T255984: Check use of global or non-global cache keys with SimpleCacheWithBagOStuff

I also created T256015: Use makeKey in CachingSiteLinkLookup and T256016: Use makeKey in CachingCommonsMediaFileNameLookup for the two classes that (currently) only have a cache for the current request, but should probably still use makeKey().