We finally have our first descriptions usages at nowiki (caused by https://no.wikipedia.org/wiki/Modul:Wikibase_signature, invoked by https://no.wikipedia.org/w/index.php?title=Mal:Wikidata_egenskapslenke&action=edit)! The description is used there to add additional context to a link's title attribute.
This requires a re-interpretation of the S usage aspect: it then means usage of "sitelinks used in the sidebar", no longer "use of any sitelinks". This means that explicit usage (via Lua) of other sitelinks needs to be tracked as O (or X).
This has been deployed. Yet there are no description usages :/
I just moved the deploy from 13:00–14:00 UTC today to 12:00–13:00 UTC, as that fits my schedule better and allows me to do more monitoring of the changes and their impact.
Sun, Oct 22
I've just added three possible implementations for this above.
Fri, Oct 20
Deployment scheduled for Monday (Oct 23) 13:00–14:00 UTC.
This is fine again, @Addshore ran extra dispatchers yesterday.
Thu, Oct 19
Well, entity:getLabel should not be looped with an explicit language given if you want to get a language in the user's language. If you manually construct the fallback chain we on our end can't detect that you actually wanted the user language… also this is not needed as getLabel() already takes fallbacks into account (for the user's language only).
Wed, Oct 18
We're already at 6,213,948 statement usages on cawiki (this is a lot).
Tue, Oct 17
Moved the deploy to 8 UTC tomorrow (October 18), per @Marostegui. Will be deployed together with T177155: Find test wiki(s) for new description usage and enable there.
Mon, Oct 16
I've scheduled the deployment for Wednesday (October 18) 18:00–19:00 UTC. I'll closely monitor the situation after, but I don't expect any more havoc.
I didn't realize earlier that I can't be around at that time, thus I moved the deploy to 08:00–09:00 UTC Wednesday (October 8).
Deployment scheduled for 13:00–14:00 UTC tomorrow (Oct 17).
Sun, Oct 15
I just ran a patched version of dumpJson along with an unmodified version (on mwdebug1001/mwdebug1002). While both servers have the same specs, the runs might not be totally comparable and I only did one run.
T178247: Use a retrieve only CachingEntityRevisionLookup for dumps should give us another few percent, as it makes loading and unserializing entities up to 15% faster. I haven't specifically tested that in the context of the dumps, but based on old profiling data, this should give us up to about 8-9% speedup for dumps (as getEntity takes 58-60% of the dump time and we made it up to 15% faster).
Sat, Oct 14
Fri, Oct 13
Brought up on enwiki: https://en.wikipedia.org/wiki/Module_talk:Wikidata#Critical_performance_improvement
The cawiki module has been fixed now, but many other wikis still have the problematic code T178114: Many Wikipedia's Wikidata module iterate over all entity claims if a Statement is searched for by property label.
Thu, Oct 12
It's going to be stable most of the time, but then (on big template/ module changes) it can very very quickly change a lot.
For cawiki: I've contacted them to fix the problem that caused this spike (https://ca.wikipedia.org/w/index.php?title=M%C3%B2dul_Discussi%C3%B3:Wikidata&oldid=18938979#Critical_performance_improvement), once that is fixed, we should be good to go ahead here.
In general, we will probably need to revisit T176273: Move the wbc_entity_usage table onto a dedicated DB shard here, especially for bigger wikis.
Please implement as I documented on gerrit: Have a feature flag which switches between X+C usages and no X usage, but dynamic tracking. That way we can enable this wiki per wiki.
Wed, Oct 11
Fixed in 9849b31626d8f2bcc31d82a39eac234446fb4de1.
cawiki had (at the time statement usage were disable again) 90,430 statement usages from just 1,118 pages. This means each of these pages had an average of about 81 statement usages (median: 90) and one page had 196 even.
Table sizes pre-deploy to cawiki and cewiki:
Even though the growth has reduced to just 1.3% this week, the dump still seems to be substantially slower… since we deployed https://gerrit.wikimedia.org/r/380628 the dumps have been at least a few hours slower :/