Page MenuHomePhabricator

LanguageTrTest::testDottedAndDotlessI fails in PHP 7.3
Closed, ResolvedPublic

Description

15:39:38 1) LanguageTrTest::testDottedAndDotlessI with data set #3 ('lcfirst', 'İ', 'upper', 'i')
15:39:38 Converting upper case 'İ' with lcfirst should give 'i'
15:39:38 Failed asserting that two strings are equal.
15:39:38 --- Expected
15:39:38 +++ Actual
15:39:38 @@ @@
15:39:38 -'i'
15:39:38 +'i̇'
15:39:38 
15:39:38 /workspace/src/tests/phpunit/languages/classes/LanguageTrTest.php:35
15:39:38 /workspace/src/tests/phpunit/MediaWikiTestCase.php:424
15:39:38 /workspace/src/maintenance/doMaintenance.php:94

(Those are not the same i.)

https://integration.wikimedia.org/ci/job/mediawiki-quibble-vendor-mysql-php73-docker/1/console

Event Timeline

Legoktm created this task.Oct 15 2018, 10:44 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 15 2018, 10:44 PM

I wonder if this is caused by some of the mbstring changes? https://github.com/php/php-src/blob/php-7.3.0RC3/UPGRADING#L186

The first is U+0069, the normal "i". The second is two characters, U+0069 U+0307, which is the normal "i" plus the combining dot above "◌̇".

It probably is due to the mbstring changes, in particular the "full case-mapping [which] may change the length of the string".

Change 467727 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[mediawiki/core@master] Update LanguageTrTest for PHP 7.3 mbstring changes

https://gerrit.wikimedia.org/r/467727

I'm not sure if Lego has any time to work on this based on no activity since November... Can anyone else from CPT pick this up @CCicalese_WMF as the last (known) blocker for PHP 7.3 support?

It doesn't sound like a lot of work is needed to finish it off

Legoktm removed Legoktm as the assignee of this task.Jun 13 2019, 7:18 PM

Sorry for cookie licking, I haven't worked on this since October :(

Or maybe Language-Team might be interested?

We'll see if we can fit this in to an upcoming sprint.

WDoranWMF triaged this task as Low priority.Jul 17 2019, 8:25 PM

Prior to PHP 7.3, the mbstring case tables were derived from UnicodeData.txt, which has U+0130 -> U+0069 as a case mapping. In 7.3+, CaseFolding.txt is used instead, which terms this mapping a special-case Turkic mapping, for language codes tr and az only. PHP 7.3's data parser thus ignores this mapping, preferring the "non-Turkic" F-type mapping U+0130 -> U+0069 U+0307. So the correct thing to do here is to implement this special mapping in LanguageTr::ucfirst().

We may also want to review the other case-folding changes in PHP 7.3, but for this task, it's a simple fix.

Change 525032 had a related patch set uploaded (by Santhosh; owner: Santhosh):
[mediawiki/core@master] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525032

santhosh claimed this task.Jul 23 2019, 7:00 AM
santhosh added a subscriber: santhosh.

So the correct thing to do here is to implement this special mapping in LanguageTr::ucfirst().

As @tstarling suggested, I implemented the 'T' type mapping to the LanguageTr in https://gerrit.wikimedia.org/r/525032

Change 525032 merged by jenkins-bot:
[mediawiki/core@master] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525032

tstarling closed this task as Resolved.Jul 24 2019, 5:06 AM

Change 525198 had a related patch set uploaded (by Reedy; owner: Santhosh):
[mediawiki/core@REL1_31] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525198

Change 525199 had a related patch set uploaded (by Reedy; owner: Santhosh):
[mediawiki/core@REL1_32] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525199

Change 525200 had a related patch set uploaded (by Reedy; owner: Santhosh):
[mediawiki/core@REL1_33] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525200

Change 525198 merged by jenkins-bot:
[mediawiki/core@REL1_31] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525198

Change 525199 merged by jenkins-bot:
[mediawiki/core@REL1_32] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525199

Change 525200 merged by jenkins-bot:
[mediawiki/core@REL1_33] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525200