$wgOut->addWikiText() doesn't run language converter
Closed, ResolvedPublic

Description

this is wrong

See screenshot. Raw code is displayed.


Version: 1.19
Severity: blocker

attachment zhwiki-converter-msgs.png ignored as obsolete

bzimport added a subscriber: wikibugs-l.
bzimport set Reference to bz34832.
MarkAHershberger created this task.Via LegacyMar 1 2012, 2:09 AM
RobLa-WMF added a comment.Via ConduitMar 1 2012, 2:14 AM

Created attachment 10138
Recent Changes screen on zh.wikipedia.org for 1.18

This is the correct screen

Attached:

RobLa-WMF added a comment.Via ConduitMar 1 2012, 2:15 AM

Created attachment 10139
Recent Changes screen on zh.wikipedia.org for 1.19

This is the wrong version. Note the -{H|zh:[公告];zh-hans:[公告];zh-hant:[公告];}- -{H|zh:[投票];zh-hans:[投票];zh-hant:[投票];}- stuff

Attached:

MarkAHershberger added a comment.Via ConduitMar 1 2012, 2:17 AM

The content of attachment 10137 has been deleted by

Mark A. Hershberger <mah@everybody.org>

who provided the following reason:

robla's screenshots are better

The token used to delete this attachment was generated at 2012-03-01 02:16:33 UTC.

tstarling added a comment.Via ConduitMar 1 2012, 2:27 AM

Appears to be due to r96812. Parser::parse() will only run language conversion if the interface message option in ParserOptions is off.

MarkAHershberger added a comment.Via ConduitMar 1 2012, 2:29 AM

adding original reporter

MarkAHershberger added a comment.Via ConduitMar 1 2012, 2:31 AM

forgot to add Liangent like I said I would.

tstarling added a comment.Via ConduitMar 1 2012, 2:39 AM

Adding the developer and reviewer to the CC list.

tstarling added a comment.Via ConduitMar 1 2012, 3:04 AM

Apparently OutputPage::addWikiText() is used for content language text without the $interface=true option in many places. So r96812 is incomplete in that it doesn't update those callers.

There is also the issue of the particularly ugly failure mode. Could we maybe decouple -{}- expansion from language conversion, choosing some sensible default expansion when language conversion is not done? For example say if I put this in MessagesEn.php:

-{fr:foo;en:bar}-

Could this just expand to "bar"?

tstarling added a comment.Via ConduitMar 1 2012, 3:04 AM

(In reply to comment #8)

Apparently OutputPage::addWikiText() is used for content language text without
the $interface=true option in many places.

I mean without $interface=false, i.e. without enabling language conversion.

liangent added a comment.Via ConduitMar 1 2012, 3:09 AM

(In reply to comment #8)

Apparently OutputPage::addWikiText() is used for content language text without
the $interface=false option in many places. So r96812 is incomplete in that it
doesn't update those callers.

I would say it's like a kind of abuse of language converter. <recentchangestext> should be $interface=true if it's fixed because it's already separated into zh-hans and zh-hant in i18n file. However on zhwiki they're mixed again and used to display something in content language...

liangent added a comment.Via ConduitMar 1 2012, 3:10 AM

Maybe a new magic word like FORCECONVERT?

siebrand added a comment.Via ConduitMar 1 2012, 8:17 AM

srwikis are also broken. See for example http://sr.wikipedia.org/sr-ec/%D0%92%D0%BB%D0%B0%D0%B4%D0%B8%D0%BC%D0%B8%D1%80_%D0%9B%D0%B5%D1%9A%D0%B8%D0%BD. Those should probably be rolled back until a fix is available, too.

SPQRobin added a comment.Via ConduitMar 1 2012, 11:35 AM

I did check which callers needed to be updated, but since the {specialpage}-summary messages are hidden by default and they're in fact in the user language, I didn't think about that. So afaics you should just use Mediawiki:Recentchanges-summary/variant, e.g. MediaWiki:Recentchanges-summary/zh-hans.

(I'd like to add that the title of this bug is a bit misleading, since addWikiText isn't supposed to run language converter because in 99% of the cases it's in the user language. Before my commits it gave very ugly results when using a custom user language and custom content variant.)

tstarling added a comment.Via ConduitMar 1 2012, 11:39 PM

List of all MediaWiki namespace pages with -{...}-:

tstarling added a comment.Via ConduitMar 2 2012, 4:03 AM

Edittools was affected by this because of a bug in MessageCache::parse() which caused the interface language option to be set in ParserOptions even if the parameter to MessageCache::parse() was false. Fixed this in 112872. Other messages may have been affected for the same reason.

liangent added a comment.Via ConduitMar 2 2012, 5:31 AM

(In reply to comment #14)

List of all MediaWiki namespace pages with -{...}-:

In this list I see this: http://zh.wikipedia.org/w/index.php?title=MediaWiki:Clearwatchlist/ja&diff=2220226&oldid=2220225 , where -{}- is used to prevent conversion. Applying Chinese conversion on Japanese can mess things up, so r96812 is meaningful and fixing some issues in some cases...

tstarling added a comment.Via ConduitMar 2 2012, 5:48 AM

For recentchangestext and edittools, I have applied fixes in trunk and tagged them for backport. I reviewed all uses of wfMsgForContent() in the MW core, and I don't think there are any others that disable language conversion inappropriately.

Most of the messages in comment #14 are for the user language. For these, perhaps fixing the wikis, either with a script or by hand, would be simplest.

liangent added a comment.Via ConduitMar 2 2012, 6:53 AM

(In reply to comment #17)

Most of the messages in comment #14 are for the user language. For these,
perhaps fixing the wikis, either with a script or by hand, would be simplest.

It isn't that simple because some messages may transclude another template (like recentchangestext).

SPQRobin added a comment.Via ConduitMar 2 2012, 7:47 PM

(In reply to comment #18)

It isn't that simple because some messages may transclude another template
(like recentchangestext).

In the case of recentchangestext, that is a message in the content language, which means language converter should work for it (fixed by Tim Starling in r112873).

(In reply to comment #16)

http://zh.wikipedia.org/w/index.php?title=MediaWiki:Clearwatchlist/ja&diff=2220226&oldid=2220225 where -{}- is used to prevent conversion.
Applying Chinese conversion on Japanese can mess things up,
so r96812 is meaningful and fixing some issues in some cases...

Using any Latin-script language on Serbian wikis, then choosing Cyrillic as variant converted a lot of the UI to Cyrillic. Imho it was a quite major bug. It still amazes me how often people use ugly hacks instead of reporting a bug.

tstarling added a comment.Via ConduitMar 5 2012, 6:42 AM

(In reply to comment #19)

Using any Latin-script language on Serbian wikis, then choosing Cyrillic as
variant converted a lot of the UI to Cyrillic. Imho it was a quite major bug.
It still amazes me how often people use ugly hacks instead of reporting a bug.

That is true. However it appears to be used in a beneficial way on the Chinese Wikipedia. Because of the inconvenient fallback sequence in MessageCache::get(), you have to create MediaWiki namespace pages in all 9 variants in order for Messages*.php to be successfully overridden. It's easier to copy and paste the same text to 9 pages, relying on the converter to work its magic, than to convert them manually.

tstarling added a comment.Via ConduitMar 5 2012, 12:17 PM

Proposed temporary patch committed in r113024.

tstarling added a comment.Via ConduitMar 8 2012, 6:22 AM

The temporary patch is deployed now, so I'm closing this bug. A bug can be opened for the issues on the converter wikis that Robin's changes were intended to fix.

SPQRobin added a comment.Via ConduitMar 9 2012, 2:29 AM

So, as an update: zh wikis are now running on the 1.18 behaviour (wgBug34832TransitionalRollback true: interface messages converted) and all other wikis on 1.19 behaviour. Only srwikipedia had -{ that needed to be removed on MediaWiki pages.

(In reply to comment #12)

srwikis are also broken. See for example
http://sr.wikipedia.org/sr-ec/%D0%92%D0%BB%D0%B0%D0%B4%D0%B8%D0%BC%D0%B8%D1%80_%D0%9B%D0%B5%D1%9A%D0%B8%D0%BD.

Those should probably be rolled back until a fix is available, too.

This is actually a different issue, see bug 35076.

siebrand added a comment.Via ConduitJan 11 2013, 4:29 PM

$wgBug34832TransitionalRollback has been removed again per merge of gerrit 43449.

Add Comment