Page MenuHomePhabricator

no upload possible when mysql database contains a "." in its name, due to 'Swift and S3 restrictions'
Closed, DuplicatePublic

Description

I recently upgraded a couple of internal wikis from mw1.16.5 directly to mw1.20.2.
Everything was working when I noticed that uploads were not working anymore. As I already knew I had to correct the rights in the Linux directories so thatthe upload would be working again (images direectory). As always this helped and the uploading was working again.
But on one wiki it did not help. After looking into the problem for a couple of hours I changed the mysql database name by coincidence from "w.wiki1" to "test-wiki". Suddenly the uploading was working again. I then found out that as soon es there is a "." in the database name uploading would stop working. The "." was not a problem in 1.16.5.
For me this is resolved as I won't use a "." in database names any more but I still wanted to point out the problem.


Version: 1.20.x
Severity: normal
OS: Linux

Details

Reference
bz44949

Event Timeline

bzimport raised the priority of this task from to Normal.Nov 22 2014, 1:24 AM
bzimport added a project: MediaWiki-Uploading.
bzimport set Reference to bz44949.
bzimport added a subscriber: Unknown Object (MLST).
SmartK created this task.Feb 13 2013, 12:44 PM

MediaWiki: 1.20.2
PHP: 5.3.10-1ubuntu3.5 (apache2handler)
MySQL: 5.5.29-0ubuntu0.12.04.1
Ubuntu: 12.04

How exactly do you upload? via Special:Upload, or via the UploadWizard extension?

This is in the MediaWiki product, so it must be Special:Upload.

I disabled all extensions for this test. And yes "Special:Upload".

Was someone of the devs able to reproduce this problem?

Sorry, nobody has tried yet.
Providing the exact database error here would be welcome (see
http://www.mediawiki.org/wiki/Manual:How_to_debug ).

Hmm, that's sad. It is very easy to reproduce as one just has to rename the database name of the wiki (inkluding a "." dot in the name). From then on the upload will not work anymore in MW.1.20.2.

Ejegg added a comment.Sep 4 2013, 4:10 AM

I can reproduce this bug, and it's not actually a database error. FileBackendStore::isValidContainerName rejects container names with periods because of 'Swift and S3 restrictions'.

The container name is prepended with the wikiId in FileBackendStore::FullContainerName.

So it seems like db names that break this regex for any reason will break uploads: /^[a-z0-9][a-z0-9-_]{0,199}$/i

I'll see where else this full container name is used, and what sort of transformation we might be able to apply that would avoid breaking S3/Swift storage but not cause wiki.db to overlap with wikidb or wiki_db.

Thanks for investigating!

Ejegg added a comment.Sep 4 2013, 12:24 PM

My pleasure! I'm new here, so I'm not sure where to propose the following fix:

For the FSFileBackend backend (local filesystem), it seems like container names are only ever used as hash keys to look up local directories. So we take the 'final' off isValidContainerName and override it in FSFileBackend to allow most anything. Then in the FileBackend constructor, we call isValidContainerName on the wikiId and throw an exception if false. It would be nice to also add a more informative error message suggesting the user set a legal wikiId in LocalSettings.php -> $wgFileBackends

aaron added a comment.Apr 22 2014, 7:01 PM

(In reply to Elliott Eggleston from comment #10)

My pleasure! I'm new here, so I'm not sure where to propose the following
fix:
For the FSFileBackend backend (local filesystem), it seems like container
names are only ever used as hash keys to look up local directories.

Not if basePath is set, which is preferred.

Maybe the solution is to support some container name encoding. The FileRepo config could have it's URLs adjusted by the admin to compensate.