Fri, Sep 22
Thu, Sep 21
@jcrespo After some discussion on https://www.mediawiki.org/wiki/Talk:Multi-Content_Revisions/Database_Schema, on https://gerrit.wikimedia.org/r/#/c/378724/4, and on the mailing list, there are a few questions that require a judgement call from a DBA. Ideally of course we would test and measure. But I'm afraid at least some of these decisions need to be made beforehand, so an assessment from experience would be very helpful.
This RFC was discussed on IRC on September 20th. Log: https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.2017-09-20-21.01.log.html
I'm still very confused about the name. "InterruptMutexManager" sounds like a manager of mutexes of interrupts. If I understand correctly, "Interrupt" in the name refers to the guarantee that the lock will be released automatically when the current request terminates, no matter how it terminates. So perhaps "AutoReleaseMutexManager" would be more descriptive. But then, will we have MutexManagers that do not guarantee this? Why not just call the interface "MutexManager"?
@Krinkle thank you for the overview!
Wed, Sep 20
All relevant patches are merged now.
@Krinkle wrote a while back:
Reopening, because the patches failed to merge. CI is failing due to a problem in the Cirrus extension T174654.
@jcrespo please note that the proposed change is for all wikibase client wikis (that is, almost all wikis). Wikidata itself also has wbc_entity_usage since it is a client of itself, but the proposed change is much more relevant for large client wikis, such as enwiki and commonswiki.
Tue, Sep 19
Mon, Sep 18
I didn't know the dashboard would save the time range in admin mode. That's kind of odd :D
Fri, Sep 15
This RFC has been scheduled for public discussion on IRC next Wednesday, September 20.
After revisiting the IRC discussion, TechCom decided to decline this RFC as proposed. However, we acknowledge that the problems listed are worth looking into. Also, the discussion turned up some interesting proposals and alternatives.
Thu, Sep 14
@Lucas_Werkmeister_WMDE, @Smalyshev: can you guys agree on something and write it down, so @Ladsgroup can implement it? I don't know which was is actually the easiest to use or the most semantically sound in RDF, and I'm busy with the db schema for MCR.
For now, let's just try to get the size of the jobs below the 4MB mark? :)
@mobrovac how about a very large number of very small jobs? e.g. a million jobs to purge a million pages from cdn?
@Pchelolo actually, can you confirm how many entries there were in the "pages" parameter? With the latest patches deployed, there should be no more than 20. Perhaps this is an old job getting retried, because it failed earlier?
@Pchelolo disabling escaping of non-ascii characters would probably reduce the size to a fourth...
If we still want to cover the "back button" use case, I think we should try and detect that situation on the client side.
Wed, Sep 13
Fix above, backport applies cleanely. Note that the fix must be merged into the wikidata build, which then has to be re-deployed (yes we know that this sucks). Perhaps @aude or @Addshore or @hoo can help. @Legoktm should also know how it works. And I guess I could manage in a pinch. Relevant documentation can be found at https://wikitech.wikimedia.org/wiki/How_to_deploy_Wikidata_code
Note that T174422: Make dbBatchSize in WikiPageUpdater configurable is related, but would not change the fact that the entire set of titles would be put into a single InjectRC job, at the moment.
InjectRCRecords batches inserts when running the job, but doesn't chop the batch up before scheduling the job.
I can easily fix that. The patch should be back-portable, too. Give me a minute...
Tue, Sep 12
Historical justification, as per Brion on IRC:
Afaics, this task as described in the summary is complete, so this ticket can be closed. Or shall we keep it open to discuss more fine grained configuration, as per my proposal above?
As per https://gerrit.wikimedia.org/r/#/c/377458/, WikiPageUpdaterDbBatchSize is now set to 20. That's probably still to high for RefreshLinksJob, and way low for UpdateHtmlCacheJob and InjectRCRecordsJob.
The batch size is already configurable, that was deployed a few weeks go. The relevant setting is WikiPageUpdaterDbBatchSize.
Mon, Sep 11
Sat, Sep 9
Thu, Sep 7
Wed, Sep 6
This RFC has been scheduled for a public discussion on IRC tonight, September 6.
Tue, Sep 5
Mon, Sep 4
It would be very nice if we had a setup that would allow us to compare different search profiles side by side, similar to the way we can now compare sql vs cirrus. Can we choose the search profile via a url parameter?
Triaging as "high" to reflect the outcome of the post-mortem with @VColeman, @demon and @greg re https://wikitech.wikimedia.org/wiki/Incident_documentation/20170721-Train-Wikidata.
Fri, Sep 1
IRC meeting happend on August 30. Log: https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.2017-08-30-21.02.log.html
Thu, Aug 31
Result: the number of clashes is minimal, and the majority of clashes seem to be unintentional. There seems to be no use case for two clashing titles to exist on a single Wiktionary. This validates the approach taken by cognate, namely, to treat certain titles as equivalent with respect to interwiki-linking.
Wed, Aug 30
Do we have the same issue with Fahrenheit as we do with Celsius?
IRC meeting on this happening NOW in the #wikimedia-office channel
Tue, Aug 29
Quick reality check: based on the above numbers, less than 2% of enwiki subscriptions are stale, and about 5% of ruwiki subscriptions are stale. That's worth investigation, but probably doesn't have a huge impact.
Pruning stale subscriptions would be nice indeed. Finding out where they come from, and fixing that, would be even better :)
Fri, Aug 25
Aug 25 2017
@aude can you help with manually testing Matt's patch? It's looking good, but it needs verification. You have central auth set up, right?
Aug 24 2017
Now let's see what the reduced batch size does. It may actually make the problem worse, but increasing the total number of jobs. Let's hope it makes it better, by reducing the time job runners are blocked...