Page MenuHomePhabricator

rebuildLocalisationCache fails with duplicate key error - problem with translation file
Open, Needs TriagePublic

Description

Hi.. Apologies if this is in the wrong place, I'm new to Wikimedia's Phabricator.

TL;DR version:

I think the following lines in translation files contain a problem:

./mediawiki-1.30.0/languages/i18n/ast.json:1406:        "rcfilters-watchlist-markSeen-button": "Marcar tolos cambios como vistos",
./mediawiki-1.30.0/languages/i18n/ba.json:1415: "rcfilters-watchlist-markSeen-button": "Бар үҙгәртеүҙе лә күрҙем тип билдәлә",
./mediawiki-1.30.0/languages/i18n/ilo.json:1378:        "rcfilters-watchlist-markSeen-button": "Markaan amin a sinuksukatan a kas nakitan",

The upper-case S in the word "markSeen" in the translation key I think should probably be lower-case.

...end TL;DR...

For me this caused the maintenance script rebuildLocalisationCache.php to exit with an error:

Query: INSERT  INTO `l10n_cache` (lc_lang,lc_key,lc_value) VALUES ('ast','.............. (very long line snipped)
Function: LCStoreDB::finishWrite
Error: 1062 Duplicate entry 'ast-messages:rcfilters-watchlist-markseen-button' for key 'PRIMARY'

Backtrace:
#0 /home/www/rememberall/rememberall/includes/libs/rdbms/database/Database.php(979): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
#1 /home/www/rememberall/rememberall/includes/libs/rdbms/database/Database.php(1589): Wikimedia\Rdbms\Database->query(string, string)
#2 /home/www/rememberall/rememberall/includes/cache/localisation/LCStoreDB.php(87): Wikimedia\Rdbms\Database->insert(string, array, string)
#3 /home/www/rememberall/rememberall/includes/cache/localisation/LocalisationCache.php(1023): LCStoreDB->finishWrite()
#4 /home/www/rememberall/rememberall/includes/cache/localisation/LocalisationCacheBulkLoad.php(100): LocalisationCache->recache(string)
#5 /home/www/rememberall/rememberall/maintenance/rebuildLocalisationCache.php(162): LocalisationCacheBulkLoad->recache(string)
#6 /home/www/rememberall/rememberall/maintenance/rebuildLocalisationCache.php(122): RebuildLocalisationCache->doRebuild(array, LocalisationCacheBulkLoad, boolean)
#7 /home/www/rememberall/rememberall/maintenance/doMaintenance.php(92): RebuildLocalisationCache->execute()
#8 /home/www/rememberall/rememberall/maintenance/rebuildLocalisationCache.php(181): require_once(string)
#9 {main}

Using a fresh test instance of MediaWiki with a binary MySQL database selected did not cause an error but did create in the database these wrong looking results:

MariaDB [mwtest]> select * from l10n_cache where lc_lang = 'ast' and lc_key like '%markseen%';
+---------+----------------------------------------------+----------------------------------------------+
| lc_lang | lc_key                                       | lc_value                                     |
+---------+----------------------------------------------+----------------------------------------------+
| ast     | messages:rcfilters-watchlist-markseen-button | s:36:"Marcar todos los cambios como vistos"; |
+---------+----------------------------------------------+----------------------------------------------+
1 row in set (0.02 sec)

MariaDB [mwtest]> select * from l10n_cache where lc_lang = 'ast' and lc_key like '%markSeen%';
+---------+----------------------------------------------+------------------------------------------+
| lc_lang | lc_key                                       | lc_value                                 |
+---------+----------------------------------------------+------------------------------------------+
| ast     | messages:rcfilters-watchlist-markSeen-button | s:32:"Marcar tolos cambios como vistos"; |
+---------+----------------------------------------------+------------------------------------------+
1 row in set (0.01 sec)

I'm guessing that the problem is caused because my 10-year-old MySQL database is text and perhaps the text case of the key is irrelevant when considering primary key uniqueness - causing a clash.

I would also bet on strange results when text using these translations is being rendered.