Page MenuHomePhabricator
Create Task
Maniphest T218860

ExtensionDependencyError FundraisingTranslateWorkflow requires Translate to be installed.
Closed, ResolvedPublic
Actions

Description

The automatic deployment to Beta-Cluster-Infrastructure his broken.

Triggered when running:

mwscript mergeMessageFileList.php --wiki="dewiktionary" --list-file="/srv/mediawiki-staging/wmf-config/extension-list" --output="/tmp/tmp.q0bUYjIqrn"

Message:

ExtensionDependencyError from line 293 of /srv/mediawiki-staging/php-master/includes/registration/ExtensionRegistry.php: FundraisingTranslateWorkflow requires Translate to be installed.
09:52:58 Started l10n-update
09:52:58 Updating ExtensionMessages-master.php
09:52:58 Last output:
Loading data from /srv/mediawiki-staging/php-master/extensions/3D/extension.json
Loading data from /srv/mediawiki-staging/php-master/extensions/AbuseFilter/extension.json
....
[55e12271489f4c0a4bc109a5] [no req]   ExtensionDependencyError from line 293 of /srv/mediawiki-staging/php-master/includes/registration/ExtensionRegistry.php: FundraisingTranslateWorkflow requires Translate to be installed.
Backtrace:
#0 /srv/mediawiki-staging/php-master/maintenance/mergeMessageFileList.php(177): ExtensionRegistry->readFromQueue(array)
#1 /srv/mediawiki-staging/multiversion/MWScript.php(100): require_once(string)
#2 {main}
09:52:58 Finished l10n-update (duration: 00m 00s)
09:52:58 Unhandled error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/scap/cli.py", line 342, in run
    exit_status = app.main(app.extra_arguments)
  File "/usr/lib/python2.7/dist-packages/scap/main.py", line 603, in main
    return super(Scap, self).main(*extra_args)
  File "/usr/lib/python2.7/dist-packages/scap/main.py", line 85, in main
    self._after_sync_common()
  File "/usr/lib/python2.7/dist-packages/scap/main.py", line 626, in _after_sync_common
    version, wikidb, self.verbose, self.config)
  File "/usr/lib/python2.7/dist-packages/scap/utils.py", line 402, in context_wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/scap/tasks.py", line 613, in update_localization_cache
    wikidb, ext_list, new_extension_messages, verbose_messagelist))
  File "/usr/lib/python2.7/dist-packages/scap/utils.py", line 402, in context_wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/scap/utils.py", line 497, in sudo_check_call
    raise subprocess.CalledProcessError(proc.returncode, cmd)
CalledProcessError: Command '/usr/local/bin/mwscript mergeMessageFileList.php --wiki="dewiktionary" --list-file="/srv/mediawiki-staging/wmf-config/extension-list" --output="/tmp/tmp.q0bUYjIqrn" ' returned non-zero exit status 255
09:52:58 scap failed: CalledProcessError Command '/usr/local/bin/mwscript mergeMessageFileList.php --wiki="dewiktionary" --list-file="/srv/mediawiki-staging/wmf-config/extension-list" --output="/tmp/tmp.q0bUYjIqrn" ' returned non-zero exit status 255 (duration: 08m 33s)

Job: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/

Details

ProjectBranchLines +/-Subject
mediawiki/extensions/FundraisingTranslateWorkflowmaster+12 -5Replace static extension registration dependency with callback hack
Customize query in gerrit

Related Objects
Search...

Event Timeline

hashar created this task.Mar 21 2019, 10:02 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 21 2019, 10:02 AM
hashar added subscribers: Ejegg, Jdforrester-WMF.Mar 21 2019, 10:04 AM

Might be related to https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/497956/ / a0553288d5902ae49e84489b89edc5740d5606e4 for T213943.

hashar added a comment.Mar 21 2019, 10:07 AM

The first broken build was on March 20th 2019 at 20:59:13 which is before that Gerrit change :/

hashar added a comment.Mar 21 2019, 10:11 AM

So it is broken and I have no idea what is going on sorry :(

MarcoAurelio awarded a token.Mar 21 2019, 10:18 AM
MarcoAurelio added a subscriber: MarcoAurelio.
hashar added a comment.Mar 21 2019, 10:26 AM

Wondering if that would cause the train to break when we next regenerate the l10n messages? T206676: 1.33.0-wmf.22 deployment blockers

hashar mentioned this in T206676: 1.33.0-wmf.22 deployment blockers.Mar 21 2019, 10:28 AM
hashar added a parent task: T206677: 1.33.0-wmf.23 deployment blockers.
Liuxinyu970226 added a subscriber: Liuxinyu970226.Mar 21 2019, 10:29 AM
hashar mentioned this in T206677: 1.33.0-wmf.23 deployment blockers.Mar 21 2019, 10:29 AM
hashar added a comment.Mar 21 2019, 10:32 AM

That also points to https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/FundraisingTranslateWorkflow/+/486345/

And https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/497956/ is supposed to be the fix.

hashar added a comment.Mar 21 2019, 10:50 AM

$wmgFundraisingTranslateWorkflow is only enabled on meta. What I guess is happening is that maintenance/mergeMessageFileList.php loads extensions from wmf-config/extension-list.json in the order listed to the file. The script does roughly:

maintenance/mergeMessageFileList.php
        # Add setup files contained in file passed to --list-file
        if ( $this->hasOption( 'list-file' ) ) {
            $extensionPaths = $this->readFile( $this->getOption( 'list-file' ) );
            $mmfl['setupFiles'] = array_merge( $mmfl['setupFiles'], $extensionPaths );
        }

foreach ( $mmfl['setupFiles'] as $fileName ) {
    // Using extension.json or skin.json
    if ( substr( $fileName, -strlen( '.json' ) ) === '.json' ) {
        $queue[$fileName] = 1;
    } else {
        require_once $fileName;
    }

if ( $queue ) { 
    $registry = new ExtensionRegistry();
    $data = $registry->readFromQueue( $queue );
    foreach ( [ 'wgExtensionMessagesFiles', 'wgMessagesDirs' ] as $var ) {
        if ( isset( $data['globals'][$var] ) ) {
            $GLOBALS[$var] = array_merge( $data['globals'][$var], $GLOBALS[$var] );
        }
    }
}

Since extensions are listed in alphabetical order, the FundraisingTranslateWorkflow extension is passed to ExtensionRegistry->readFromQueue() before Translate and bails out. The extension has:

mediawiki/extensions/FundraisingTranslateWorkflow/extension.json
{
    "requires": {
        "extensions": {
            "Translate": "*"
        }
    }
}

Introduced by https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/FundraisingTranslateWorkflow/+/486345/ and committed on Wed Mar 20 20:42:53 2019 +0000.

Note that Translate has not been migrated to extension registry yet. Thus we extension-list has:

wmf-config/extension-list
IP/extensions/FundraisingTranslateWorkflow/extension.json
$IP/extensions/Translate/Translate.php

I would assume extension registry to hold a bit before enforcing the Translate requirement, but maybe the lack of extension.json in Translate causes extension registry to not know about it?

That is all I have for now. Require assistance from someone more knowledgeable about the registry mecanism :-(

hashar added a comment.Mar 21 2019, 10:56 AM

A very basic attempt at reproducing the issue on my local setup:

$ php maintenance/shell.php
>>> require_once 'extensions/Translate/Translate.php';
=> 1
>>> $registry = new ExtensionRegistry();
=> ExtensionRegistry {#1082}
>>> $queue = [ 'extensions/FundraisingTranslateWorkflow/extension.json' => 1 ];
=> [
     "extensions/FundraisingTranslateWorkflow/extension.json" => 1,
   ]
>>> $registry->readFromQueue( $queue );
ExtensionDependencyError with message 'FundraisingTranslateWorkflow requires Translate to be installed.
>>> $registry->isLoaded( 'Translate' );
=> false
>>>
Nikerabbit added a subscriber: Nikerabbit.Mar 21 2019, 1:40 PM
• Addshore added a subscriber: WMDE-Fisch.Mar 21 2019, 2:55 PM
• Tonina_Zhelyazkova_WMDE added a subscriber: • Tonina_Zhelyazkova_WMDE.Mar 21 2019, 2:58 PM
Jdforrester-WMF added a comment.Mar 21 2019, 4:35 PM

Oh, yeah, you can't use proper dependencies for extensions that haven't migrated yet. One moment, I'll write you a patch.

gerritbot added a comment.Mar 21 2019, 4:42 PM

Change 498149 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/extensions/FundraisingTranslateWorkflow@master] Replace static extension registration dependency with callback hack

https://gerrit.wikimedia.org/r/498149

gerritbot added a project: Patch-For-Review.Mar 21 2019, 4:42 PM
WMDE-Fisch added a subscriber: • Tim_WMDE.Mar 21 2019, 5:06 PM
gerritbot added a comment.Mar 21 2019, 6:19 PM

Change 498149 merged by jenkins-bot:
[mediawiki/extensions/FundraisingTranslateWorkflow@master] Replace static extension registration dependency with callback hack

https://gerrit.wikimedia.org/r/498149

Jdforrester-WMF closed this task as Resolved.Mar 21 2019, 6:42 PM
Jdforrester-WMF claimed this task.

Now fixed: https://integration.wikimedia.org/ci/view/Beta/job/beta-scap-eqiad/242213/console passes l10n-update.

ReleaseTaggerBot added a project: MW-1.33-notes (1.33.0-wmf.23; 2019-03-26).Mar 21 2019, 7:01 PM
Liuxinyu970226 removed a subscriber: Liuxinyu970226.Mar 26 2019, 12:14 AM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL