Page MenuHomePhabricator

Clean up current images with wrong SHA1
Closed, ResolvedPublic

Description

Extremely strange bug report from IRC:

<zscout370> Error deleting file: A non-identical file already exists at

mwstore://local-backend/local-deleted/m/s/x/msx6tm4d7koft50ig332cbh3yuds9mb.svg.
....uhh...what is this?

<hexmode> url?
<zscout370> hexmode, trying to fix

http://commons.wikimedia.org/wiki/File:Flag_of_Kazakhstan.svg due
to someone uploading at least 10 times

<hexmode> zscout370: that is a funky bug
<zscout370> and the history is also borked
<zscout370> and when I am trying to delete the page to clear the history, I

get that bug

<hexmode> zscout370: can I have your wikiname and post your comments

into a bug that I create?

<zscout370> zscout370 is my Commons username (and sysop)
<zscout370> hexmode, for the bug link

http://commons.wikimedia.org/w/index.php?title=File:Flag_of_Kazakhstan.svg&action=delete&oldimage=20120227155653!Flag_of_Kazakhstan.svg

<zscout370> <filedeleteerror-short-old>


Version: 1.19
Severity: normal

Details

Reference
bz34755

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 12:17 AM
bzimport set Reference to bz34755.

Marking 1.19 until I find out I'm wrong.

Created attachment 10117
Screenshot

<zscout370> that is all the versions that are at Flag of Kazakhstan.svg (I was

trying to fix the image since someone uploaded so many images in
about 3 minutes, with many reverts) so I was trying to fix that
and that was how I encountered the bug.

<zscout370> now

http://upload.wikimedia.org/wikipedia/commons/d/d3/Flag_of_Kazakhstan.svg
is showing as 404, but the image is still there?

<hexmode> you mean the thumbs are there?
<zscout370> yes
<zscout370> it looks like an image is there, thumbs are there, but when you

click on it it goes 404

<zscout370> and if you need it, I was on Firefox 10.0.2, Win7 Ultimate

Attached:

zscout370 wrote:

Comment on attachment 10117
Screenshot

The screenshot can be seen at http://www.flickr.com/photos/zscout370/6789632196/

zscout370 wrote:

The file was uploaded several times by a user today, and while in the process of fixing the image, I encountered that error. It will not let me delete the old versions and deleting the top version will only delete the physical page (but not the files themselves). The image also shows as 404 but all thumbnails and data are being generated in the file history.

Dumping the DB sha1 and the actual sha1 via script, I see:
string(31) "hmag9lxxuekjy8xsj3e133qmkbpryfb"
string(31) "hmag9lxxuekjy8xsj3e133qmkbpryfb"

string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"
string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"

string(31) "msx6tm4d7koft50ig332cbh3yuds9mb"
string(31) "msx6tm4d7koft50ig332cbh3yuds9mb"

string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"
string(31) "msx6tm4d7koft50ig332cbh3yuds9mb"

string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"
string(31) "6lo5nwy70mn4g5w1er8ys8syfi2hk67"

string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"
string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"

string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"
string(31) "j4ogmam1gcokm1xf577a8dze72y66ab"

string(31) "msx6tm4d7koft50ig332cbh3yuds9mb"
string(31) "msx6tm4d7koft50ig332cbh3yuds9mb"

Notice the two pairs in the middle that due not match. This is a side-effect of bug 17057.

Even if we don't solve bug 17057, there should be a better failure mode. This looks pretty rare, though, so setting milestone appropriately.

  • Bug 34959 has been marked as a duplicate of this bug. ***

Aaron and I discussed this, and nearly marked this as a duplicate of bug 17057. However, Aaron doesn't feel comfortable closing bug 17057 until we actually know that new images with the wrong SHA1 are no longer being created. So, we're reappropriating this bug to be "clean everything up to this point", which basically will involve writing and running a cleanup script. After that script is run, this bug will be closed, but bug 17057 will remain open.

Using the 1.19wmf1 deployment milestone, since this isn't waiting for 1.20 in any meaningful way.

aaron added a comment.Apr 3 2012, 8:30 PM

*** Bug 35656 has been marked as a duplicate of this bug. ***

zscout370 wrote:

I am now seeing this again in another file: http://commons.wikimedia.org/w/index.php?title=Special:Undelete&action=submit

"Undelete failed; someone else may have undeleted the page first.

Undeletion will not be performed if it will result in the top page or file revision being partially deleted. In such cases, you must uncheck or unhide the newest deleted revision.

Error undeleting file: A non-identical file already exists at mwstore://local-backend/local-public/8/85/Flag_of_Belarus.svg."

When trying to upload: "Unexpected non-MediaWiki exception encountered, of type "SyntaxException" exception 'SyntaxException' with message 'Object name not set.' in /usr/local/apache/common-local/php-1.19/extensions/SwiftCloudFiles/php-cloudfiles-1.7.10/cloudfiles.php:1767 Stack trace: #0 /usr/local/apache/common-local/wmf-config/swift.php(48): CF_Container->delete_object('') #1 [internal function]: wmfPurgeBackendThumbCache(Object(LocalFile), false) #2 /usr/local/apache/common-local/php-1.19/includes/Hooks.php(216): call_user_func_array('wmfPurgeBackend...', Array) #3 /usr/local/apache/common-local/php-1.19/includes/GlobalFunctions.php(3823): Hooks::run('LocalFilePurgeT...', Array) #4 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(732): wfRunHooks('LocalFilePurgeT...', Array) #5 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(972): LocalFile->purgeThumbnails() #6 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(914): LocalFile->recordUpload2('20120410003419!...', 'Updated flag ht...', false, Array, false, Object(User)) #7 /usr/local/apache/common-local/php-1.19/includes/upload/UploadBase.php(573): LocalFile->upload('/tmp/phpb0HOKQ', 'Updated flag ht...', false, 1, Array, false, Object(User)) #8 /usr/local/apache/common-local/php-1.19/includes/specials/SpecialUpload.php(439): UploadBase->performUpload('Updated flag ht...', false, true, Object(User)) #9 /usr/local/apache/common-local/php-1.19/includes/specials/SpecialUpload.php(174): SpecialUpload->processUpload() #10 /usr/local/apache/common-local/php-1.19/includes/SpecialPageFactory.php(476): SpecialUpload->execute(NULL) #11 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(263): SpecialPageFactory::executePath(Object(Title), Object(RequestContext)) #12 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(593): MediaWiki->performRequest() #13 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(503): MediaWiki->main() #14 /usr/local/apache/common-local/php-1.19/index.php(58): MediaWiki->run() #15 /usr/local/apache/common-local/live-1.5/index.php(3): require('/usr/local/apac...') #16 {main} #0 /usr/local/apache/common-local/wmf-config/swift.php(48): CF_Container->delete_object('') #1 [internal function]: wmfPurgeBackendThumbCache(Object(LocalFile), false) #2 /usr/local/apache/common-local/php-1.19/includes/Hooks.php(216): call_user_func_array('wmfPurgeBackend...', Array) #3 /usr/local/apache/common-local/php-1.19/includes/GlobalFunctions.php(3823): Hooks::run('LocalFilePurgeT...', Array) #4 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(732): wfRunHooks('LocalFilePurgeT...', Array) #5 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(972): LocalFile->purgeThumbnails() #6 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(914): LocalFile->recordUpload2('20120410003419!...', 'Updated flag ht...', false, Array, false, Object(User)) #7 /usr/local/apache/common-local/php-1.19/includes/upload/UploadBase.php(573): LocalFile->upload('/tmp/phpb0HOKQ', 'Updated flag ht...', false, 1, Array, false, Object(User)) #8 /usr/local/apache/common-local/php-1.19/includes/specials/SpecialUpload.php(439): UploadBase->performUpload('Updated flag ht...', false, true, Object(User)) #9 /usr/local/apache/common-local/php-1.19/includes/specials/SpecialUpload.php(174): SpecialUpload->processUpload() #10 /usr/local/apache/common-local/php-1.19/includes/SpecialPageFactory.php(476): SpecialUpload->execute(NULL) #11 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(263): SpecialPageFactory::executePath(Object(Title), Object(RequestContext)) #12 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(593): MediaWiki->performRequest() #13 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(503): MediaWiki->main() #14 /usr/local/apache/common-local/php-1.19/index.php(58): MediaWiki->run() #15 /usr/local/apache/common-local/live-1.5/index.php(3): require('/usr/local/apac...') #16 {main}"

(In reply to comment #10)

[[commons:File:Flag of Belarus.svg]] now disappeared, the file description page stating “No file by this name exists, but you can upload it (even though there still is https://upload.wikimedia.org/wikipedia/commons/8/85/Flag_of_Belarus.svg). If a file used to exist, try to purge this page's cache”

And trying to purge the page leads to:

Unexpected non-MediaWiki exception encountered, of type "SyntaxException"
exception 'SyntaxException' with message 'Object name not set.' in /usr/local/apache/common-local/php-1.19/extensions/SwiftCloudFiles/php-cloudfiles-1.7.10/cloudfiles.php:1767
Stack trace:
#0 /usr/local/apache/common-local/wmf-config/swift.php(48): CF_Container->delete_object('')
#1 [internal function]: wmfPurgeBackendThumbCache(Object(LocalFile), false)
#2 /usr/local/apache/common-local/php-1.19/includes/Hooks.php(216): call_user_func_array('wmfPurgeBackend...', Array)
#3 /usr/local/apache/common-local/php-1.19/includes/GlobalFunctions.php(3823): Hooks::run('LocalFilePurgeT...', Array)
#4 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(732): wfRunHooks('LocalFilePurgeT...', Array)
#5 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(681): LocalFile->purgeThumbnails(Array)
#6 /usr/local/apache/common-local/php-1.19/includes/WikiFilePage.php(164): LocalFile->purgeCache(Array)
#7 [internal function]: WikiFilePage->doPurge()
#8 /usr/local/apache/common-local/php-1.19/includes/Article.php(1786): call_user_func_array(Array, Array)
#9 [internal function]: Article->__call('doPurge', Array)
#10 /usr/local/apache/common-local/php-1.19/includes/actions/PurgeAction.php(51): ImagePage->doPurge()
#11 /usr/local/apache/common-local/php-1.19/includes/actions/PurgeAction.php(69): PurgeAction->onSubmit(Array)
#12 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(484): PurgeAction->show()
#13 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(278): MediaWiki->performAction(Object(ImagePage))
#14 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(593): MediaWiki->performRequest()
#15 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(503): MediaWiki->main()
#16 /usr/local/apache/common-local/php-1.19/index.php(58): MediaWiki->run()
#17 /usr/local/apache/common-local/live-1.5/index.php(3): require('/usr/local/apac...')
#18 {main}

#0 /usr/local/apache/common-local/wmf-config/swift.php(48): CF_Container->delete_object('')
#1 [internal function]: wmfPurgeBackendThumbCache(Object(LocalFile), false)
#2 /usr/local/apache/common-local/php-1.19/includes/Hooks.php(216): call_user_func_array('wmfPurgeBackend...', Array)
#3 /usr/local/apache/common-local/php-1.19/includes/GlobalFunctions.php(3823): Hooks::run('LocalFilePurgeT...', Array)
#4 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(732): wfRunHooks('LocalFilePurgeT...', Array)
#5 /usr/local/apache/common-local/php-1.19/includes/filerepo/file/LocalFile.php(681): LocalFile->purgeThumbnails(Array)
#6 /usr/local/apache/common-local/php-1.19/includes/WikiFilePage.php(164): LocalFile->purgeCache(Array)
#7 [internal function]: WikiFilePage->doPurge()
#8 /usr/local/apache/common-local/php-1.19/includes/Article.php(1786): call_user_func_array(Array, Array)
#9 [internal function]: Article->__call('doPurge', Array)
#10 /usr/local/apache/common-local/php-1.19/includes/actions/PurgeAction.php(51): ImagePage->doPurge()
#11 /usr/local/apache/common-local/php-1.19/includes/actions/PurgeAction.php(69): PurgeAction->onSubmit(Array)
#12 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(484): PurgeAction->show()
#13 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(278): MediaWiki->performAction(Object(ImagePage))
#14 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(593): MediaWiki->performRequest()
#15 /usr/local/apache/common-local/php-1.19/includes/Wiki.php(503): MediaWiki->main()
#16 /usr/local/apache/common-local/php-1.19/index.php(58): MediaWiki->run()
#17 /usr/local/apache/common-local/live-1.5/index.php(3): require('/usr/local/apac...')
#18 {main}

I would say this is quite a severe problem, given e.g. [[Special:WhatLinksHere/File:Flag_of_Belarus.svg]]

aaron added a comment.Apr 10 2012, 6:09 PM

I've changed swift.php to skip "" files in the listings. I have no idea why the swift listings have empty files.

Moving this down in priority only because Aaron is working on what he believes is the root cause (bug 36132), and I want to move this down below that on his list of stuff to do. With any luck, this will automatically get fixed as a result of that.

(removing 1.19wmf1 deployment blocker, 1.19wmf1 is already deployed)
(moving from 1.19.0 release to 1.19.x, 1.19.0 has been released)

Is this a MediaWiki bug or something to be fixed in Wikimedia databases?

I think it is in MediaWiki, but just asking to be sure.

So this would be in the form of a maintenance script? Or part of update.php? (both?)

aaron added a comment.May 9 2012, 7:36 PM

This bug is for running a script.

aaron added a comment.Jun 5 2012, 9:00 PM

I've starting run this on commons and a few large wikis. It may take a few weeks to finish.

aaron added a comment.Jun 25 2012, 5:21 PM

(In reply to comment #17)

I've starting run this on commons and a few large wikis. It may take a few
weeks to finish.

Finally finished on commons after the second attempt (other wikis done already).

Gilles raised the priority of this task from Medium to Unbreak Now!.Dec 4 2014, 10:25 AM
Gilles added a project: Multimedia.
Gilles moved this task from Untriaged to Done on the Multimedia board.
Gilles lowered the priority of this task from Unbreak Now! to Medium.Dec 4 2014, 11:20 AM