Page MenuHomePhabricator

Cannot delete files with non-standard extensions on MySQL in a strict mode
Closed, ResolvedPublicBUG REPORT

Description

Steps to replicate the issue (include links if applicable):

  • Set up MediaWiki with MySQL
  • Enable file upload and add additional allowed extension
    • $wgEnableUploads = true;
    • $wgFileExtensions = [ 'png', 'gif', 'jpg', 'jpeg', 'webp', 'ttf' ];
  • If working in development mode, disable strict warnings $wgDBStrictWarnings = false;, otherwise enable strict SQL Mode $wgSQLMode = 'STRICT_ALL_TABLES';.
  • Upload file with a non-standard extension, eg. .ttf.
  • You can notice that the file is saved in the image table with ''/0 value in img_major_mime column
  • Now try to delete the file

What happens?:

There is an error, and the file cannot be deleted:

Wikimedia\Rdbms\DBQueryError from line 1627 of /includes/libs/rdbms/database/Database.php: Error 1265: Data truncated for column 'fa_major_mime' at row 1
Function: LocalFileDeleteBatch::doDBInserts
Query: INSERT INTO `filearchive` ...

MediaWiki tried to move the record from the image table to the filearchive table, but it was impossible because fa_major_mime is an enum, and ''/0 value is not allowed when MySQL is configured to work in a strict mode.

The img_major_mime column in the image table has a similar enum, but image upload works, because the code appends IGNORE to the INSERT statement, and MySQL converts the not-allowed enum value to ''/0`.

What should have happened instead?:
The file should be deleted.

Software version (skip for WMF-hosted wikis like Wikipedia):
Latest code from the master, but also older versions, eg. 1.39.

I submitted two patches that could fix this issue:

  1. Stop producing data with invalid enum values, by using an unknown value for cases when the calculated mime is different than allowed by the database schema: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/992148
  2. Allow deletion of files with invalid img_major_mime values, by using IGNORE in the INSERT to filearchive table statement. It could alternatively be replaced by a database update fixing bad records: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/992150

Event Timeline

Change 992148 had a related patch set uploaded (by apasternak; author: apasternak):

[mediawiki/core@master] filerepo: Fix img_major_mime for files with a non-standard extensions

https://gerrit.wikimedia.org/r/992148

Change 992150 had a related patch set uploaded (by apasternak; author: apasternak):

[mediawiki/core@master] filerepo: Allow deletion of files with invalid img_major_mime

https://gerrit.wikimedia.org/r/992150

Change 992148 merged by jenkins-bot:

[mediawiki/core@master] filerepo: Fix img_major_mime for files with a non-standard extensions

https://gerrit.wikimedia.org/r/992148

Change 993004 had a related patch set uploaded (by Paladox; author: apasternak):

[mediawiki/core@REL1_41] filerepo: Fix img_major_mime for files with a non-standard extensions

https://gerrit.wikimedia.org/r/993004

Change 993005 had a related patch set uploaded (by Paladox; author: apasternak):

[mediawiki/core@REL1_40] filerepo: Fix img_major_mime for files with a non-standard extensions

https://gerrit.wikimedia.org/r/993005

Change 993769 had a related patch set uploaded (by Reedy; author: apasternak):

[mediawiki/core@REL1_39] filerepo: Fix img_major_mime for files with a non-standard extensions

https://gerrit.wikimedia.org/r/993769

Change 993806 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/core@master] MimeAnalyzer: Add @since to isValidMajorMimeType

https://gerrit.wikimedia.org/r/993806

Change 993004 merged by jenkins-bot:

[mediawiki/core@REL1_41] filerepo: Fix img_major_mime for files with a non-standard extensions

https://gerrit.wikimedia.org/r/993004

Change 993005 merged by jenkins-bot:

[mediawiki/core@REL1_40] filerepo: Fix img_major_mime for files with a non-standard extensions

https://gerrit.wikimedia.org/r/993005

Change 993769 merged by jenkins-bot:

[mediawiki/core@REL1_39] filerepo: Fix img_major_mime for files with a non-standard extensions

https://gerrit.wikimedia.org/r/993769

Change 993806 merged by jenkins-bot:

[mediawiki/core@master] MimeAnalyzer: Add @since to isValidMajorMimeType

https://gerrit.wikimedia.org/r/993806

Change 994231 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/core@REL1_41] MimeAnalyzer: Add @since to isValidMajorMimeType

https://gerrit.wikimedia.org/r/994231

Change 994232 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/core@REL1_40] MimeAnalyzer: Add @since to isValidMajorMimeType

https://gerrit.wikimedia.org/r/994232

Change 994233 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/core@REL1_39] MimeAnalyzer: Add @since to isValidMajorMimeType

https://gerrit.wikimedia.org/r/994233

Change 994233 merged by jenkins-bot:

[mediawiki/core@REL1_39] MimeAnalyzer: Add @since to isValidMajorMimeType

https://gerrit.wikimedia.org/r/994233

Change 994231 merged by jenkins-bot:

[mediawiki/core@REL1_41] MimeAnalyzer: Add @since to isValidMajorMimeType

https://gerrit.wikimedia.org/r/994231

Change 994232 merged by jenkins-bot:

[mediawiki/core@REL1_40] MimeAnalyzer: Add @since to isValidMajorMimeType

https://gerrit.wikimedia.org/r/994232

Change 992150 merged by jenkins-bot:

[mediawiki/core@master] filerepo: Allow deletion of files with invalid img_major_mime

https://gerrit.wikimedia.org/r/992150

TK-999 assigned this task to apasternak.
TK-999 triaged this task as Low priority.