Page MenuHomePhabricator

Nuke should batch the deletions
Closed, ResolvedPublic

Description

@Reedy suggests that, in order to avoid frequent DB timeouts, Nuke should, instead of submitting all pages to be deleted at once, do so in batches. I can confirm that sometimes we run into errors like the one recently reported:

[WpiFVQpAME4AAAjJP1UAAABT] 2018-03-01 22:57:41: Lock wait timeout exceeded; try restarting transaction (10.64.48.23)
SELECT page_latest  FROM 'page' WHERE page_id = '66950405' AND page_namespace = '6' AND page_title = '31zufe2rx5L._AC_US400_QL65_(final_inflatable).jpg' LIMIT 1 FOR UPDATE

(in this case, 150+ file deletions at once).

Event Timeline

Bawolff updated the task description. (Show Details)

Thanks for posting the Logstash trace.

Ladsgroup added a subscriber: Ladsgroup.

I'd do it because why not.

So not even deleteBatch.php batches the deletions. It basically doesn't exist in mediawiki. What should happen here is that Nuke should queue deletion jobs instead.

Change 715323 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/extensions/Nuke@master] Queue deletion job instead of trying to delete the pages

https://gerrit.wikimedia.org/r/715323

Change 715323 merged by jenkins-bot:

[mediawiki/extensions/Nuke@master] Queue deletion job instead of trying to delete the pages

https://gerrit.wikimedia.org/r/715323

Legoktm added a subscriber: Legoktm.

Per https://www.mediawiki.org/wiki/Topic:Wja1wqjt0mq36oqz this is taking a significant amount of time to delete pages, maybe it needs to be made higher priority in the job queue?

Note that another instance where mass deletions were queued (on mediawiki.org) did not progress (nothing actually deleted) for ~13hrs (after which I gave up and just resorted to manually deleting things one by one). I'm not sure it's entirely a priority issue.

I wonder if instead of using the job queue if we tried to do the deletion client-side with some concurrency, like Twinkle's delete batch functionality.

The problem is also reported as T294686, a fix has been proposed there and it will be deployed on Monday.

The patch for it got deployed. Is it fixed now?

The patch for it got deployed. Is it fixed now?

I just tried to nuke a page on plwiktionary and it got instantly deleted.

Ditto. Several moderately large batches today, all were deleted promptly.

The problem is also reported as T294686, a fix has been proposed there and it will be deployed on Monday.

(Can't see the fix though---access denied---why secret?)

Ditto. Several moderately large batches today, all were deleted promptly.

The problem is also reported as T294686, a fix has been proposed there and it will be deployed on Monday.

(Can't see the fix though---access denied---why secret?)

It got deployed as a security fix. Someone from Security team will open it once they are happy it's resolved and backported which may be the next Security Announcement.

Probably nothing to backport.

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Nuke/+/715323 landed in master for wmf/1.38.0-wmf.6, and wasn't backported...

Change 736284 had a related patch set uploaded (by Reedy; author: Amir Sarabadani):

[mediawiki/extensions/Nuke@REL1_37] Queue deletion job instead of trying to delete the pages

https://gerrit.wikimedia.org/r/736284

Change 736288 had a related patch set uploaded (by Reedy; author: Amir Sarabadani):

[mediawiki/extensions/Nuke@REL1_36] Queue deletion job instead of trying to delete the pages

https://gerrit.wikimedia.org/r/736288

Change 736291 had a related patch set uploaded (by Reedy; author: Amir Sarabadani):

[mediawiki/extensions/Nuke@REL1_35] Queue deletion job instead of trying to delete the pages

https://gerrit.wikimedia.org/r/736291

Change 736284 merged by jenkins-bot:

[mediawiki/extensions/Nuke@REL1_37] Queue deletion job instead of trying to delete the pages

https://gerrit.wikimedia.org/r/736284

Change 736291 merged by jenkins-bot:

[mediawiki/extensions/Nuke@REL1_35] Queue deletion job instead of trying to delete the pages

https://gerrit.wikimedia.org/r/736291

Change 736288 merged by jenkins-bot:

[mediawiki/extensions/Nuke@REL1_36] Queue deletion job instead of trying to delete the pages

https://gerrit.wikimedia.org/r/736288

This is done now.