Deleting some specific files is not working on Commons
Closed, DuplicatePublicPRODUCTION ERROR
Actions

Description

Hi,

when trying to delete the file listed in this deletion request, using Gadget-DelReqHandler, I got the following message:

"DelReqHandler
API request failed (internal_api_error_DBUnexpectedError): [8fd8c356-437f-48aa-9c2f-5bb6527649a4] Caught exception of type Wikimedia\Rdbms\DBUnexpectedError"

when trying to delete the same file using Gadget-instantDelete, I got the result:

"Deletion failed"

when trying to delete the file using the link "Delete all" available in the file history of the file page, I got:

"Error deleting file: An unknown error occurred in storage backend "local-multiwrite". "

The same issues occurs with the file listed in this other deletion request, and for each case we are several administrators to fail to delete those files.
What seems to me an important common point between these two files, is that the both files has two upload log entries, see 1 and 2, and in each case ther is no images uploaded for the first entry.

Details

Stack Trace

[Exception Wikimedia\Rdbms\DBUnexpectedError] (/srv/mediawiki/php-1.36.0-wmf.2/includes/libs/rdbms/database/Database.php:4335) No atomic section is open (got LocalFile::lockingTransaction)
  #0 /srv/mediawiki/php-1.36.0-wmf.2/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->endAtomic(string)
  #1 /srv/mediawiki/php-1.36.0-wmf.2/includes/libs/rdbms/database/DBConnRef.php(629): Wikimedia\Rdbms\DBConnRef->__call(string, array)
  #2 /srv/mediawiki/php-1.36.0-wmf.2/includes/filerepo/file/LocalFile.php(2451): Wikimedia\Rdbms\DBConnRef->endAtomic(string)
  #3 /srv/mediawiki/php-1.36.0-wmf.2/includes/filerepo/file/LocalFile.php(2469): LocalFile->unlock()
  #4 /srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiDelete.php(92): LocalFile->__destruct()
  #5 /srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiMain.php(1593): ApiDelete->execute()
  #6 /srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiMain.php(529): ApiMain->executeAction()
  #7 /srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiMain.php(500): ApiMain->executeActionWithErrorHandling()
  #8 /srv/mediawiki/php-1.36.0-wmf.2/api.php(90): ApiMain->execute()
  #9 /srv/mediawiki/php-1.36.0-wmf.2/api.php(45): wfApiMain()
  #10 /srv/mediawiki/w/api.php(3): require(string)
  #11 {main}
Caused by: [Exception ApiUsageException] (/srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiBase.php:1460) An unknown error occurred in storage backend "local-multiwrite".
  #0 /srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiDelete.php(92): ApiBase->dieStatus(Status)
reedy@mwlog1001:/srv/mw-log$ grep 8fd8c356 exception.log -A 25
2020-08-02 20:22:04 [8fd8c356-437f-48aa-9c2f-5bb6527649a4] mw1394 commonswiki 1.36.0-wmf.2 exception ERROR: [8fd8c356-437f-48aa-9c2f-5bb6527649a4] /w/api.php   Wikimedia\Rdbms\DBUnexpectedError from line 4335 of /srv/mediawiki/php-1.36.0-wmf.2/includes/libs/rdbms/database/Database.php: No atomic section is open (got LocalFile::lockingTransaction) {"exception_id":"8fd8c356-437f-48aa-9c2f-5bb6527649a4","exception_url":"/w/api.php","caught_by":"entrypoint"} 
[Exception Wikimedia\Rdbms\DBUnexpectedError] (/srv/mediawiki/php-1.36.0-wmf.2/includes/libs/rdbms/database/Database.php:4335) No atomic section is open (got LocalFile::lockingTransaction)
  #0 /srv/mediawiki/php-1.36.0-wmf.2/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->endAtomic(string)
  #1 /srv/mediawiki/php-1.36.0-wmf.2/includes/libs/rdbms/database/DBConnRef.php(629): Wikimedia\Rdbms\DBConnRef->__call(string, array)
  #2 /srv/mediawiki/php-1.36.0-wmf.2/includes/filerepo/file/LocalFile.php(2451): Wikimedia\Rdbms\DBConnRef->endAtomic(string)
  #3 /srv/mediawiki/php-1.36.0-wmf.2/includes/filerepo/file/LocalFile.php(2469): LocalFile->unlock()
  #4 /srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiDelete.php(92): LocalFile->__destruct()
  #5 /srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiMain.php(1593): ApiDelete->execute()
  #6 /srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiMain.php(529): ApiMain->executeAction()
  #7 /srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiMain.php(500): ApiMain->executeActionWithErrorHandling()
  #8 /srv/mediawiki/php-1.36.0-wmf.2/api.php(90): ApiMain->execute()
  #9 /srv/mediawiki/php-1.36.0-wmf.2/api.php(45): wfApiMain()
  #10 /srv/mediawiki/w/api.php(3): require(string)
  #11 {main}
Caused by: [Exception ApiUsageException] (/srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiBase.php:1460) An unknown error occurred in storage backend "local-multiwrite".
  #0 /srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiDelete.php(92): ApiBase->dieStatus(Status)
  #1 /srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiMain.php(1593): ApiDelete->execute()
  #2 /srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiMain.php(529): ApiMain->executeAction()
  #3 /srv/mediawiki/php-1.36.0-wmf.2/includes/api/ApiMain.php(500): ApiMain->executeActionWithErrorHandling()
  #4 /srv/mediawiki/php-1.36.0-wmf.2/api.php(90): ApiMain->execute()
  #5 /srv/mediawiki/php-1.36.0-wmf.2/api.php(45): wfApiMain()
  #6 /srv/mediawiki/w/api.php(3): require(string)
  #7 {main}

Related Objects

Mentioned In: T244567: Some files cannot be deleted "Error deleting file: An unknown error occurred in storage backend "local-multiwrite". " (due to missing physical file for old image entries)
Mentioned Here: T244567: Some files cannot be deleted "Error deleting file: An unknown error occurred in storage backend "local-multiwrite". " (due to missing physical file for old image entries)

Event Timeline

ChristianFerrer created this task.Aug 2 2020, 8:42 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 2 2020, 8:42 PM

ChristianFerrer updated the task description. (Show Details)Aug 2 2020, 8:44 PM

Reedy changed the subtype of this task from "Task" to "Production Error".Aug 2 2020, 8:46 PM

Reedy added a project: Wikimedia-production-error.

Reedy edited Stack Trace. (Show Details)

Files uploaded in 2013 with two upload entries at the same minute.

Reedy removed a project: MediaWiki-extensions-UploadBlacklist.Aug 2 2020, 8:49 PM

Platonides added a project: MediaWiki-Uploading.Aug 2 2020, 9:14 PM

DannyS712 subscribed.Aug 3 2020, 5:46 AM

Just for the info, before to start the report here, I had began a dicussion in Commons:Administrators' noticeboard#Commons:Deletion requests/File:Liga de béisbol del Valle.jpg (solid link), and I just learned that other files with potentially the same issues are categorized there.

Jdlrobson moved this task from Untriaged to Older on the Wikimedia-production-error board.Sep 16 2020, 6:49 PM

Krinkle renamed this task from Deleting some specific files is not working to Deleting some specific files is not working on Commons.Sep 16 2020, 6:50 PM

Krinkle edited projects, added MediaWiki-File-management, MediaWiki-Page-deletion, Structured Data Engineering; removed MediaWiki-Uploading.

Restricted Application added projects: Commons, Structured-Data-Backlog. · View Herald TranscriptSep 16 2020, 6:50 PM

For SDE: Until more information is available, it's hard to tell whether this is rooted in general page deletion logic or the Swift/multimedia backend. If the former, you may want to collab with CPT on this one.

How did that exception chaining happen? It doesn't make any sense.
Also, it seems like the LocalFile destructor was invoked on this line. I was under the impression that GC happens right when the object becomes unreferenced, did that change at some point? I think some of our code implicitly assumes it is the case.

CBogen moved this task from Triage to Multimedia on the Structured-Data-Backlog board.Sep 21 2020, 4:29 PM

Is this task a duplicate of T244567, or are they two different bugs manifesting in the same error message?

@AntiCompositeNumber I believe you're right. The top error message is different, but the one from this task is actually two layers indirect and the bottom exception is indeed the same!

Krinkle closed this task as a duplicate of T244567: Some files cannot be deleted "Error deleting file: An unknown error occurred in storage backend "local-multiwrite". " (due to missing physical file for old image entries).Oct 4 2020, 1:52 AM

Krinkle mentioned this in T244567: Some files cannot be deleted "Error deleting file: An unknown error occurred in storage backend "local-multiwrite". " (due to missing physical file for old image entries).

Deleting some specific files is not working on CommonsClosed, DuplicatePublicPRODUCTION ERRORActions

Description

Details

Related Objects

Event Timeline

Deleting some specific files is not working on Commons
Closed, DuplicatePublicPRODUCTION ERROR
Actions