Page MenuHomePhabricator

Deleted files can remain on swift due to race conditions
Open, MediumPublic

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Can i ask You ? how to delete any file is Dangerous on wikimedia commons ?

Actually, that file belongs to https://commons.wikimedia.org/wiki/File:20170615014039!Younes.webm, and it's gone after the file is undeleted then redeleted

zhuyifei1999 renamed this task from uploads.wm.o commons archive 20170615014039!Adsalm.webm visible despite file deleted on Commons to Deleted files can remain on swift dure to race conditions.Jun 17 2017, 12:27 AM
zhuyifei1999 renamed this task from Deleted files can remain on swift dure to race conditions to Deleted files can remain on swift due to race conditions.
zhuyifei1999 set Security to Software security bug.
zhuyifei1999 added a project: acl*security.
zhuyifei1999 changed the visibility from "Public (No Login Required)" to "Custom Policy".
zhuyifei1999 updated the task description. (Show Details)
zhuyifei1999 removed a subscriber: Younes19956.

Logs for the bot:
{P5591}

In both cases there are internal_api_error_LocalFileLockError.

Are these files gone now, or is there something still left to do for this specific bug?

Try find out how we are supposed to make sure race conditions will not leave a file on swift? Such fast reuploads & deletes are still happening, and may leave such files anytime.

This phenomenon is similar to T109331 and T133821 to end users that the permanent links remain visible after deletion. The difference is that the linked tickets describe the issue where varnish fails to purge the data and reload from swift, but this one is that the files actually remain on swift (tested with X-Wikimedia-Debug and and adding query parameters; if I'm wrong, please close as duplicate).

The reason why undelete then delete did not help until after purge by Reedy was probably because of the linked tickets.

FWIW the exception on mw side for the file above looks like this (id WUHllwpAAEkAAG5N0gEAAABV)

#0 /srv/mediawiki/php-1.30.0-wmf.5/includes/filerepo/file/LocalFile.php(1753): LocalFile->lock()
#1 /srv/mediawiki/php-1.30.0-wmf.5/includes/FileDeleteForm.php(203): LocalFile->delete(string, boolean, User)
#2 /srv/mediawiki/php-1.30.0-wmf.5/includes/api/ApiDelete.php(164): FileDeleteForm::doDelete(Title, LocalFile, NULL, string, boolean, User, NULL)
#3 /srv/mediawiki/php-1.30.0-wmf.5/includes/api/ApiDelete.php(75): ApiDelete::deleteFile(WikiFilePage, User, NULL, string, boolean, NULL)
#4 /srv/mediawiki/php-1.30.0-wmf.5/includes/api/ApiMain.php(1578): ApiDelete->execute()
#5 /srv/mediawiki/php-1.30.0-wmf.5/includes/api/ApiMain.php(545): ApiMain->executeAction()
#6 /srv/mediawiki/php-1.30.0-wmf.5/includes/api/ApiMain.php(516): ApiMain->executeActionWithErrorHandling()
#7 /srv/mediawiki/php-1.30.0-wmf.5/api.php(83): ApiMain->execute()
#8 /srv/mediawiki/w/api.php(3): include(string)
#9 {main}
<godog> zhuyifei1999_: looks like a bug on mw side though, I suggest you add some mediawiki projects too

5+ years later, should we make this public? The issue showed up again in T328875.

I see no particular issue with doing so from a swift POV.

5+ years later, should we make this public?

I don't mind. The WP0 upload abuse the long history now.

zhuyifei1999 changed the visibility from "Custom Policy" to "Public (No Login Required)".Feb 14 2023, 1:32 PM