Page MenuHomePhabricator

LanguageTrTest::testDottedAndDotlessI fails in PHP 7.3
Closed, ResolvedPublic

Description

15:39:38 1) LanguageTrTest::testDottedAndDotlessI with data set #3 ('lcfirst', 'İ', 'upper', 'i')
15:39:38 Converting upper case 'İ' with lcfirst should give 'i'
15:39:38 Failed asserting that two strings are equal.
15:39:38 --- Expected
15:39:38 +++ Actual
15:39:38 @@ @@
15:39:38 -'i'
15:39:38 +'i̇'
15:39:38 
15:39:38 /workspace/src/tests/phpunit/languages/classes/LanguageTrTest.php:35
15:39:38 /workspace/src/tests/phpunit/MediaWikiTestCase.php:424
15:39:38 /workspace/src/maintenance/doMaintenance.php:94

(Those are not the same i.)

https://integration.wikimedia.org/ci/job/mediawiki-quibble-vendor-mysql-php73-docker/1/console

Event Timeline

Legoktm created this task.Oct 15 2018, 10:44 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 15 2018, 10:44 PM

I wonder if this is caused by some of the mbstring changes? https://github.com/php/php-src/blob/php-7.3.0RC3/UPGRADING#L186

The first is U+0069, the normal "i". The second is two characters, U+0069 U+0307, which is the normal "i" plus the combining dot above "◌̇".

It probably is due to the mbstring changes, in particular the "full case-mapping [which] may change the length of the string".

Change 467727 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[mediawiki/core@master] Update LanguageTrTest for PHP 7.3 mbstring changes

https://gerrit.wikimedia.org/r/467727

I'm not sure if Lego has any time to work on this based on no activity since November... Can anyone else from CPT pick this up @CCicalese_WMF as the last (known) blocker for PHP 7.3 support?

It doesn't sound like a lot of work is needed to finish it off

Legoktm removed Legoktm as the assignee of this task.Jun 13 2019, 7:18 PM

Sorry for cookie licking, I haven't worked on this since October :(

Or maybe Language-Team might be interested?

We'll see if we can fit this in to an upcoming sprint.

WDoranWMF triaged this task as Low priority.Jul 17 2019, 8:25 PM
WDoranWMF moved this task from Inbox to Next on the Platform Team Workboards (Clinic Duty Team) board.

Prior to PHP 7.3, the mbstring case tables were derived from UnicodeData.txt, which has U+0130 -> U+0069 as a case mapping. In 7.3+, CaseFolding.txt is used instead, which terms this mapping a special-case Turkic mapping, for language codes tr and az only. PHP 7.3's data parser thus ignores this mapping, preferring the "non-Turkic" F-type mapping U+0130 -> U+0069 U+0307. So the correct thing to do here is to implement this special mapping in LanguageTr::ucfirst().

We may also want to review the other case-folding changes in PHP 7.3, but for this task, it's a simple fix.

Change 525032 had a related patch set uploaded (by Santhosh; owner: Santhosh):
[mediawiki/core@master] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525032

santhosh claimed this task.Jul 23 2019, 7:00 AM
santhosh added a subscriber: santhosh.

So the correct thing to do here is to implement this special mapping in LanguageTr::ucfirst().

As @tstarling suggested, I implemented the 'T' type mapping to the LanguageTr in https://gerrit.wikimedia.org/r/525032

Change 525032 merged by jenkins-bot:
[mediawiki/core@master] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525032

tstarling closed this task as Resolved.Jul 24 2019, 5:06 AM

Change 525198 had a related patch set uploaded (by Reedy; owner: Santhosh):
[mediawiki/core@REL1_31] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525198

Change 525199 had a related patch set uploaded (by Reedy; owner: Santhosh):
[mediawiki/core@REL1_32] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525199

Change 525200 had a related patch set uploaded (by Reedy; owner: Santhosh):
[mediawiki/core@REL1_33] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525200

Change 525198 merged by jenkins-bot:
[mediawiki/core@REL1_31] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525198

Change 525199 merged by jenkins-bot:
[mediawiki/core@REL1_32] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525199

Change 525200 merged by jenkins-bot:
[mediawiki/core@REL1_33] Update LanguageTrTest::testDottedAndDotlessI for PHP 7.3

https://gerrit.wikimedia.org/r/525200

Change 467727 abandoned by Legoktm:
Update LanguageTrTest for PHP 7.3 mbstring changes

Reason:
Already fixed

https://gerrit.wikimedia.org/r/467727