Page MenuHomePhabricator

update.php is super-slow thanks to populateChangeTagDef.php
Closed, ResolvedPublic

Description

Due to it inexplicably sleeping for 10 seconds in between each tag updated.

Event Timeline

Anomie created this task.Oct 9 2018, 7:15 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 9 2018, 7:15 PM

hmm, if there is already tags to be populated, it makes sense to have it to avoid read-only in non-WMF prod.

We're getting close to wanting to release 1.32, and this is marked as a blocker. Is there a reason not to remove this sleep, at least as the default behavior?

We're getting close to wanting to release 1.32, and this is marked as a blocker. Is there a reason not to remove this sleep, at least as the default behavior?

I mentioned the reason above, if you run this in production (not WMF), it would cause the whole database go read-only because of number of the writes will be super big. One thing is to reduce it to 2 seconds, it'll help but not much.

Anomie added a comment.EditedDec 14 2018, 6:05 PM

Sorry, I misinterpreted your earlier comment.

If you run this in WMF production, you can pass the --sleep option. Whereas with update.php you're a smaller site and probably want it done as quickly as possible to minimize the site downtime involved with running update.php.

Although, why does the waitForReplication() call not avoid overloading the database without the arbitrary sleep?

Although, why does the waitForReplication() call not avoid overloading the database without the arbitrary sleep?

It has it too, This is additional because waitForReplication() doesn't care about non-main replicas like the second datacenter, dbstore, etc. in WMF.

Change 479907 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/core@master] Set default of sleep time in populateChangeTagDef to zero

https://gerrit.wikimedia.org/r/479907

Change 479907 merged by jenkins-bot:
[mediawiki/core@master] Set default of sleep time in populateChangeTagDef to zero

https://gerrit.wikimedia.org/r/479907

Anomie closed this task as Resolved.Dec 17 2018, 7:47 PM
Anomie assigned this task to Ladsgroup.
Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptDec 17 2018, 7:47 PM

Change 480447 had a related patch set uploaded (by Legoktm; owner: Ladsgroup):
[mediawiki/core@REL1_32] Set default of sleep time in populateChangeTagDef to zero

https://gerrit.wikimedia.org/r/480447

Change 480447 merged by jenkins-bot:
[mediawiki/core@REL1_32] Set default of sleep time in populateChangeTagDef to zero

https://gerrit.wikimedia.org/r/480447