Page MenuHomePhabricator

Address Voice and Tone issues in Translate
Open, MediumPublic

Description

Current status

We've updated the configuration options and added a wrapper class to provide backward compatibility for the old options for at least one MLEB release. We've update translatewiki.net configuration to use these new configuration options. The next MLEB release will be in July, 2021, we can remove the old options completely.

Original report

Targets
    Occurrences of '(white|black)[ \-]?list' in Directory /Users/reedy/PhpstormProjects/mediawiki/extensions/Translate
Found Occurrences  (120 usages found)
    extensions  (120 usages found)
        Translate  (120 usages found)
            api  (3 usages found)
                ApiQueryMessageCollection.php  (3 usages found)
                    44 global $wgTranslateBlacklist;
                    73 if ( isset( $wgTranslateBlacklist[ $check ][ $languageCode ] ) ) {
                    75 $reason = $wgTranslateBlacklist[ $check ][ $languageCode ];
            data  (2 usages found)
                group-yaml-schema.yaml  (2 usages found)
                    109 whitelist:
                    113 blacklist:
            ffs  (13 usages found)
                PremadeMediawikiExtensionGroups.php  (6 usages found)
                    171 'whitelist' => [],
                    172 'blacklist' => [],
                    179 $conf['LANGUAGES']['whitelist'][] = $tag;
                    181 $conf['LANGUAGES']['blacklist'][] = $tag;
                    183 $conf['LANGUAGES']['blacklist'] = '*';
                    184 $conf['LANGUAGES']['whitelist'][] = $tag;
                SimpleFFS.php  (7 usages found)
                    320 * Remove blacklisted authors.
                    327 global $wgTranslateAuthorBlacklist;
                    333 $blacklisted = false;
                    334 foreach ( $wgTranslateAuthorBlacklist as $rule ) {
                    339 $blacklisted = false;
                    342 $blacklisted = true;
                    347 if ( $blacklisted ) {
            i18n  (4 usages found)
                core  (4 usages found)
                    en.json  (2 usages found)
                        299 "right-translate-sandboxaction": "Execute actions whitelisted for sandboxed users",
                        300 "action-translate-sandboxaction": "execute actions whitelisted for sandboxed users",
                    qqq.json  (2 usages found)
                        60 "translate-page-disabled": "Parameters:\n* $1 - reason that the language has been blacklisted",
                        61 "translate-language-disabled": "Shown when someone requests a blacklisted language. [{{FULLURL:Special:Translate|group=mediawiki&language=zh-tw}} Example].",
            messagegroups  (26 usages found)
                MessageGroup.php  (2 usages found)
                    166 * Get all the translatable languages for a group, considering the whitelisting
                    167 * and blacklisting.
                MessageGroupBase.php  (22 usages found)
                    318 * Get all the translatable languages for a group, considering the whitelisting
                    319 * and blacklisting.
                    323 global $wgTranslateBlacklist;
                    334 if ( isset( $lists['blacklist'] ) ) {
                    335 $blacklist = $lists['blacklist'];
                    336 if ( $blacklist === '*' ) {
                    337 // All languages blacklisted
                    339 } elseif ( is_array( $blacklist ) ) {
                    340 foreach ( $blacklist as $code ) {
                    345 // Treat lack of explicit blacklist the same as blacklisting everything. This way,
                    346 // when one defines only whitelist, it means that only those languages are allowed.
                    350 // DWIM with $wgTranslateBlacklist, e.g. languages in that list should not unexpectedly
                    351 // be enabled when a whitelist is used to whitelist any language.
                    354 if ( isset( $wgTranslateBlacklist[ $check ] ) ) {
                    355 foreach ( array_keys( $wgTranslateBlacklist[ $check ] ) as $blacklistedCode ) {
                    356 unset( $codes[ $blacklistedCode ] );
                    361 if ( isset( $lists['whitelist'] ) ) {
                    362 $whitelist = $lists['whitelist'];
                    363 if ( $whitelist === '*' ) {
                    364 // All languages whitelisted (except $wgTranslateBlacklist)
                    366 } elseif ( is_array( $whitelist ) ) {
                    367 foreach ( $whitelist as $code ) {
                MessageGroupOld.php  (2 usages found)
                    327 * Get all the translatable languages for a group, considering the whitelisting
                    328 * and blacklisting.
            specials  (10 usages found)
                SpecialLanguageStats.php  (2 usages found)
                    459 * is blacklisted or hidden by filters.
                    470 if ( $this->table->isBlacklisted( $groupId, $this->target ) !== null ) {
                SpecialMessageGroupStats.php  (1 usage found)
                    149 if ( $table->isBlacklisted( $this->target, $code ) !== null ) {
                SpecialSupportedLanguages.php  (7 usages found)
                    35 'TranslateAuthorBlacklist',
                    203 $blacklist = $this->options->get( 'TranslateAuthorBlacklist' );
                    210 $blacklisted = false;
                    211 foreach ( $blacklist as $rule ) {
                    216 $blacklisted = false;
                    219 $blacklisted = true;
                    224 if ( $blacklisted ) {
            src  (8 usages found)
                Synchronization  (4 usages found)
                    ExportTranslationsMaintenanceScript.php  (4 usages found)
                        193 $whitelist = $group->getTranslatableLanguages();
                        204 // Do not export languages that are blacklisted (or not whitelisted).
                        205 // Also check that whitelist is not null, which means that all
                        207 if ( is_array( $whitelist ) && !isset( $whitelist[$lang] ) ) {
                Validation  (4 usages found)
                    ValidationRunner.php  (4 usages found)
                        204 global $wgTranslateCheckBlacklist;
                        206 if ( $wgTranslateCheckBlacklist === false ) {
                        212 $wgTranslateCheckBlacklist,
                        213 'checkBlacklist'
            tag  (3 usages found)
                PageTranslationHooks.php  (3 usages found)
                    977 // And finally check whether the language is not included in whitelist
                    1006 $whitelist = [
                    1011 if ( in_array( $action, $whitelist ) ) {
            tests  (21 usages found)
                phpunit  (21 usages found)
                    data  (2 usages found)
                        check-blacklist.php  (1 usage found)
                            3 $checkBlacklist = [
                        messageindexdata.ser  (1 usage found)
                            1 fcnz-oynpxyvfg";a:4:{i:0;s:17:"ext-spamblacklist";i:1;R:801;i:2;R:3237;i:3;R:3238;}s:16:"8:fcnz-juvgryvfg";a:4:{i:0;R:4329;i:1;R:801;i:2;R:3237;i:3
                    ffs  (2 usages found)
                        MediaWikiExtensionsTest.php  (2 usages found)
                            37 $this->assertArrayHasKey( 'foo', $languages, 'Whitelisted language is available' );
                            38 $this->assertArrayNotHasKey( 'bar', $languages, 'Blacklisted language is not available' );
                    Validation  (6 usages found)
                        ValidationRunnerTest.php  (6 usages found)
                            33 $this->setMwGlobals( 'wgTranslateCheckBlacklist', false );
                            171 'wgTranslateCheckBlacklist' => __DIR__ . '/../data/check-blacklist.php'
                            188 'warnings or errors are filtered as per check-blacklist.'
                            195 'warnings or errors are filtered as per check-blacklist only for specific language code.'
                            202 'warnings or errors are filtered as per check-blacklist.'
                            209 'warnings or errors are filtered as per check-blacklist only for specific language code.'
                    BlackListTest.php  (11 usages found)
                        10 * Unit tests for blacklisting/whitelisting languages for a message group
                        12 class BlackListTest extends MediaWikiTestCase {
                        46 public function testAllBlackList() {
                        49 'blacklist' => '*',
                        56 public function testAllWhiteList() {
                        59 'whitelist' => '*',
                        66 public function testWhiteListOverrideBlackList() {
                        69 'whitelist' => [ 'en', 'hi', 'ta' ],
                        70 'blacklist' => [ 'ta' ],
                        78 public function testSomeBlackList() {
                        81 'blacklist' => [ 'or', 'hi' ],
            utils  (12 usages found)
                StatsTable.php  (9 usages found)
                    260 public function isBlacklisted( $groupId, $code ) {
                    261 global $wgTranslateBlacklist;
                    263 $blacklisted = null;
                    272 if ( isset( $wgTranslateBlacklist[$check] ) && isset( $wgTranslateBlacklist[$check][$code] ) ) {
                    273 $blacklisted = $wgTranslateBlacklist[$check][$code];
                    276 if ( $blacklisted !== null ) {
                    284 $blacklisted = true;
                    289 $blacklisted = true;
                    292 return $blacklisted;
                TranslateSandbox.php  (3 usages found)
                    316 * Whitelisting for certain API modules. See also enforcePermissions.
                    324 $whitelist = [
                    333 if ( $module->isWriteMode() && !in_array( $class, $whitelist, true ) ) {
            extension.json  (6 usages found)
                921 "TranslateBlacklist": {
                923 parsed as wikitext.\n\nExample:\n'TranslateBlacklist': { '*': { 'en': 'English is the source language.' }, 'core': { 'mul': 'Not a real language.' }, 'ext
                926 "TranslateCheckBlacklist": {
                928 "description": "File containing checks that are to be skipped. See https://gerrit.wikimedia.org/g/translatewiki/+/master/groups/check-blacklist.php for example.\nSince 2012-10-15.",
                931 "TranslateAuthorBlacklist": {
                933 : "Two-dimensional array of rules that blacklists certain authors from appearing in the exports. This is useful for keeping bots and people doing maintenance
            HISTORY  (7 usages found)
                112 - Improved hiding of blacklisted groups on Special:LanguageStats.
                117 - Allow black listing and white listing of languages in YAML configuration. YAML
                119   optional subsections whitelist and blacklist, which take an array of language
                120   codes. If a subsection is not specified, the white/blacklist value will
                121   default to *, meaning all known languages. Whitelist overrides blacklist.
                716 * $wgTranslateBlacklist added to prevent edits to certain language/group combinations
                721 * added $wgTranslateAuthorBlacklist
            MessageGroups.php  (2 usages found)
                848 $whitelist = $group->getTranslatableLanguages();
                849 if ( is_array( $whitelist ) && !isset( $whitelist[$language] ) ) {
            TranslateEditAddons.php  (3 usages found)
                47 global $wgTranslateBlacklist;
                78 if ( isset( $wgTranslateBlacklist[$check][$langCode] ) ) {
                79 $reason = $wgTranslateBlacklist[$check][$langCode];

Details

ProjectBranchLines +/-Subject
translatewikimaster+2 -2
translatewikimaster+22 -22
mediawiki/extensions/Translatemaster+4 -4
mediawiki/extensions/Translatemaster+23 -15
mediawiki/extensions/Translatemaster+1 -1
mediawiki/extensions/Translatemaster+144 -42
translatewikimaster+1 -1
translatewikimaster+17 -17
translatewikimaster+3 -3
translatewikimaster+2 -2
operations/mediawiki-configmaster+1 -1
mediawiki/extensions/Translatemaster+4 -4
mediawiki/extensions/Translatemaster+13 -13
mediawiki/extensions/Translatemaster+6 -6
mediawiki/extensions/Translatemaster+19 -16
mediawiki/extensions/Translatemaster+18 -18
mediawiki/extensions/Translatemaster+4 -4
mediawiki/extensions/Translatemaster+2 -2
mediawiki/extensions/Translatemaster+59 -70
Show related patches Customize query in gerrit

Event Timeline

Change 675717 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):
[mediawiki/extensions/Translate@master] Replace black/white list with exclusion/inclusion list in code

https://gerrit.wikimedia.org/r/675717

Change 675725 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):
[mediawiki/extensions/Translate@master] Replace black/white list in config description and language files

https://gerrit.wikimedia.org/r/675725

Change 676273 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/Translate@master] Rename config TranslateCheckBlacklist to TranslateCheckExclusionList

https://gerrit.wikimedia.org/r/676273

Change 676274 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[translatewiki@master] Rename wgTranslateCheckBlacklist to wgTranslateCheckExclusionList

https://gerrit.wikimedia.org/r/676274

Change 676277 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[translatewiki@master] Rename wgTranslateAuthorBlacklist to wgTranslateAuthorExclusionList

https://gerrit.wikimedia.org/r/676277

Change 676278 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/Translate@master] Rename config TranslateAuthorBlacklist to TranslateAuthorExclusionList

https://gerrit.wikimedia.org/r/676278

Change 676850 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/Translate@master] Rename config TranslateBlacklist to TranslateExclusionList

https://gerrit.wikimedia.org/r/676850

Change 676851 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[translatewiki@master] Rename wgTranslateBlacklist to wgTranslateExclusionList

https://gerrit.wikimedia.org/r/676851

Change 676909 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[operations/mediawiki-config@master] Rename wgTranslateBlacklist to wgTranslateExclusionList

https://gerrit.wikimedia.org/r/676909

Change 677106 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/Translate@master] Replace black/whitelist with exclusion/inclusionlist in group config

https://gerrit.wikimedia.org/r/677106

Change 677107 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[translatewiki@master] Update group configuration to use inclusion/exclusion list

https://gerrit.wikimedia.org/r/677107

Change 677125 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/Translate@master] Address voice issues in HISTORY

https://gerrit.wikimedia.org/r/677125

I've submitted a series a patches that replaces usage of blacklist with exclusionList and whitelist with inclusionList. That should take care of all occurrences of (white|black)[ \-]?list in Translate but some more work will be needed for translatewiki.net

Change 675717 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] Replace black/white list with exclusion/inclusion list in code

https://gerrit.wikimedia.org/r/675717

Change 677125 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] Address voice issues in HISTORY

https://gerrit.wikimedia.org/r/677125

Change 697731 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/Translate@master] Add ConfigManager to wrap configuration requests

https://gerrit.wikimedia.org/r/697731

Change 699141 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/Translate@master] Add support for include/exclude in group LANGUAGES config

https://gerrit.wikimedia.org/r/699141

Change 699142 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/Translate@master] Remove blacklist/whitelist from MessageIndex test file

https://gerrit.wikimedia.org/r/699142

Change 699334 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/Translate@master] Replace black/white list in language files

https://gerrit.wikimedia.org/r/699334

Change 699334 abandoned by Abijeet Patro:

[mediawiki/extensions/Translate@master] Replace black/white list in language files

Reason:

Will submit again.

https://gerrit.wikimedia.org/r/699334

Change 699335 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/Translate@master] Replace black/white list in language files

https://gerrit.wikimedia.org/r/699335

Change 677106 abandoned by Abijeet Patro:

[mediawiki/extensions/Translate@master] Replace black/whitelist with exclusion/inclusionlist in group config

Reason:

In favor of Ib80749ced89192b066e70c24e1897c1aae1d1a9f

https://gerrit.wikimedia.org/r/677106

Change 676850 abandoned by Abijeet Patro:

[mediawiki/extensions/Translate@master] Rename config TranslateBlacklist to TranslateDisabledTargetLanguages

Reason:

In favor of Ifb6419785252d10bf6f43e09fb20080aaaa33fab

https://gerrit.wikimedia.org/r/676850

Change 676278 abandoned by Abijeet Patro:

[mediawiki/extensions/Translate@master] Rename config TranslateAuthorBlacklist to TranslateAuthorExclusionList

Reason:

In favor of Ifb6419785252d10bf6f43e09fb20080aaaa33fab

https://gerrit.wikimedia.org/r/676278

Change 676273 abandoned by Abijeet Patro:

[mediawiki/extensions/Translate@master] Rename config TranslateCheckBlacklist to TranslateValidationExclusionFile

Reason:

In favor of Ifb6419785252d10bf6f43e09fb20080aaaa33fab

https://gerrit.wikimedia.org/r/676273

Change 675725 abandoned by Abijeet Patro:

[mediawiki/extensions/Translate@master] Replace black/white list in language files

Reason:

In favor of Ib80749ced89192b066e70c24e1897c1aae1d1a9f

https://gerrit.wikimedia.org/r/675725

Change 676909 abandoned by Abijeet Patro:

[operations/mediawiki-config@master] Rename wgTranslateBlacklist to wgTranslateExclusionList

Reason:

Will submit a new patch.

https://gerrit.wikimedia.org/r/676909

Change 677107 abandoned by Abijeet Patro:

[translatewiki@master] Update group configuration to use inclusion/exclusion list

Reason:

Will submit a new patch.

https://gerrit.wikimedia.org/r/677107

Change 676274 abandoned by Abijeet Patro:

[translatewiki@master] Rename wgTranslateCheckBlacklist to wgTranslateValidationExclusionFile

Reason:

Will submit a new patch.

https://gerrit.wikimedia.org/r/676274

Change 676277 abandoned by Abijeet Patro:

[translatewiki@master] Rename wgTranslateAuthorBlacklist to wgTranslateAuthorExclusionList

Reason:

Will submit a new patch.

https://gerrit.wikimedia.org/r/676277

Change 676851 abandoned by Abijeet Patro:

[translatewiki@master] Rename wgTranslateBlacklist to wgTranslateExclusionList

Reason:

Will submit a new patch.

https://gerrit.wikimedia.org/r/676851

Change 697731 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] Add ConfigHelper to wrap configuration requests

https://gerrit.wikimedia.org/r/697731

Change 699141 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] Add support for include/exclude in group LANGUAGES config

https://gerrit.wikimedia.org/r/699141

Change 699142 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] Remove blacklist/whitelist from MessageIndex test file

https://gerrit.wikimedia.org/r/699142

Change 699335 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] Replace black/white list in language files

https://gerrit.wikimedia.org/r/699335

Change 700417 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[translatewiki@master] Use new configuration variables

https://gerrit.wikimedia.org/r/700417

Change 700418 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[translatewiki@master] Replace blacklist/whitelist in group LANGUAGES config

https://gerrit.wikimedia.org/r/700418

Change 700417 merged by jenkins-bot:

[translatewiki@master] Use new configuration variables and files

https://gerrit.wikimedia.org/r/700417

Change 700418 merged by jenkins-bot:

[translatewiki@master] Replace blacklist/whitelist in group LANGUAGES config

https://gerrit.wikimedia.org/r/700418