Page MenuHomePhabricator

Updater race condition when using replication
Open, NormalPublic

Description

Using update.php on a wiki with replication has the potential of hitting a race condition where the master has finished an update operation, but the slave(s) hasn't; and then a query hits the slave but fails due to missing columns etc.

This can be shown more clearly by broken replication:

[20:37:15] <jcl> I'm having trouble upgrading MW from 1.20.8 to 1.23: during the update.php script, I'm getting a db query error: "select page_namespace, etc. from page where ..." from the function Title::newFromID is erroring with "Unknown column page_content_model" in 'field list' -- I've seen a few similar results from googling but nothing I've done so far has helped. Any suggestions?
<INSERT DEBUGGING>
[20:59:59] <jcl> well there you go, the slave threads aren't running, gr...

In a lot of cases, update.php may not be the correct tool for the job for updating these wikis, but it happens.

Maybe it should bail if more than 1 server. Or maybe more sensibly, queries to the slaves should be disabled.

To go along with this, some wfWaitForSlave() type calls should probably be added between updater operations too.


Version: unspecified
Severity: normal

Details

Reference
bz66887

Event Timeline

bzimport raised the priority of this task from to Normal.Nov 22 2014, 3:26 AM
bzimport added a project: MediaWiki-Installer.
bzimport set Reference to bz66887.
bzimport added a subscriber: Unknown Object (MLST).
Reedy created this task.Jun 20 2014, 8:12 PM

Change 141049 had a related patch set uploaded by Reedy:
Add wfWaitForSlaves() call in DatabaseUpdater::runUpdates()

https://gerrit.wikimedia.org/r/141049

Reedy added a comment.Jun 20 2014, 8:18 PM

21:04:21] <bawolff> I think update.php is written with the assumption that anyone using it has a db small enough to not need replication

^ In that case, it's see bug 1

Paladox set Security to None.Jun 6 2015, 11:40 PM

Change 141049 had a related patch set uploaded (by Paladox):
Add wfWaitForSlaves() call in DatabaseUpdater::runUpdates()

https://gerrit.wikimedia.org/r/141049

Change 141049 abandoned by Reedy:
Add wfWaitForSlaves() call in DatabaseUpdater::runUpdates()

https://gerrit.wikimedia.org/r/141049

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 23 2015, 12:58 PM

Change 141049 restored by Reedy:
Add wfWaitForSlaves() call in DatabaseUpdater::runUpdates()

https://gerrit.wikimedia.org/r/141049

Change 141049 merged by jenkins-bot:
Add wfWaitForSlaves() call in DatabaseUpdater::runUpdates()

https://gerrit.wikimedia.org/r/141049