Page MenuHomePhabricator

wfShellExec destroys unicode characters if no utf-8 locale available on the system
Open, LowPublic

Description

Author: dasch

Description:
When switching from PHPs Image converting to ImageMagick it does not work an I get the following error

Fehler beim Erstellen des Vorschaubildes: convert: unable to open image `/var/www/vhosts/wecowi.de/httpdocs/images/1/16/TBBT_5-06_Sheldons_Mutter_zu_Besuch_-_groe_Runde.jpg': No such file or directory @ blob.c/OpenBlob/2480.
convert: missing an image filename `/var/www/vhosts/wecowi.de/httpdocs/images/thumb/1/16/TBBT_5-06_Sheldons_Mutter_zu_Besuch_-_groe_Runde.jpg/300px-TBBT_5-06_Sheldons_Mutter_zu_Besuch_-_groe_Runde.jpg' @ convert.c/ConvertImageCommand/2838.

The name of the File is TBBT 5-06 Sheldons Mutter zu Besuch - große Runde.jpg
convert seams to ignore the ß in the filename

Permissions are okay, because when turning imagemagick off it works
So it seams that the commands generated for imagemagick are not UTF-8 safe


Version: 1.20.x
Severity: normal

Details

Reference
bz31752

Event Timeline

bzimport raised the priority of this task from to Low.Nov 21 2014, 11:51 PM
bzimport set Reference to bz31752.
bzimport added a subscriber: Unknown Object (MLST).

dasch wrote:

And at the same time the converting of images with only ASCII characters works with ImageMagick

dasch wrote:

Well the problem appears when locals on the server are not in UTF-8
I added a HowTo to mediawiki
http://www.mediawiki.org/wiki/Manual:Errors_and_symptoms#Image_Thumbnails_not_working_and.2For_appearing

But maybe this should be checked somehow

dasch wrote:

Maybe something like a diagnosis special page would be helpful?

Bryan.TongMinh wrote:

Is there something left that we can do to fix this error on the MediaWiki side? It's more like a server config problem.

Perhaps that the installer can trigger a warning if the locale is set wrong.

dasch wrote:

Yes that would be a good idea
I also started this extension
http://www.mediawiki.org/w/index.php?title=Special:Code/MediaWiki&path=%2Ftrunk%2Fextensions%2FDiagnosis
Which could help to find possible problems

brion added a comment.Oct 16 2011, 6:24 PM

Looks like wfInitShellLocale() tries to force a UTF-8 locale to work around http://bugs.php.net/bug.php?id=45132 where escapeshellarg() destroys the non-ASCII chars if your locale isn't properly set to UTF-8. (r41379)

I presume that's not working on a system that has missing or broken UTF-8 locales? (ick!)

IMHO any non-UTF8 locale should be quietly laid to rest in this day and age, but it's nicer to just not have to run into the problem. ;)

If this is only a bug in escapeshellarg(), maybe we should just give up and implement it ourselves -- wfEscapeShellArg() already does that for Windows, where the stock escapeshellarg() is wildly incorrect.

dasch wrote:

I think the best way would be to check if UTF-8 is in locales under Linux and go the "windows way" if it's not available
The point is, that on many minimal installation of linux UTF-8 is not set up

For regonition of such problems still a SpecialPage like the suggested in my extension would be helpful

Something that other PHP-Software like Joomla or Piwik already have

brion added a comment.Oct 19 2011, 6:59 PM

Why bother messing with the locale stuff is the only thing we need it for is to work around a broken escapeshellarg()?

(Offhand I'm not sure where these non-UTF8 Linux systems are coming from, I guess I'm just spoiled by using mainstream distros for the last decade, which seem to all default to UTF-8. ;)

dasch wrote:

I got this non-UTF-8 Linux with my V-Server, it's just a minimalistic installation of Ubuntu. That's not about the distro, but about the typ of installation.

Jdforrester-WMF moved this task from Untriaged to Backlog on the Multimedia board.Sep 4 2015, 6:14 PM
Restricted Application added subscribers: Steinsplitter, Matanya, Aklapper. · View Herald TranscriptSep 4 2015, 6:14 PM