Page MenuHomePhabricator

Issue with inter-extension-dependencies in CI tests
Closed, ResolvedPublic

Description

For BlueSpice we have created REL1_31_dev branches in all repos, that we use to prepare the upcoming minor release. Unfortunately it seems that there is sometimes an issue with that branch in the CI tests.

e.g. a commit to extension "BlueSpiceVisualEditorConnector" fails with

10:46:40 Fatal error: Class 'BlueSpice\Extension' not found in /workspace/src/extensions/BlueSpiceVisualEditorConnector/src/Extension.php on line 34

https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php70-docker/5447/console

The class BlueSpice\Extension is defined in the "BlueSpiceFoundation" extension, which is declared as a dependency in zuul/parameter_functions.py. From the log I can see that it gets installed.

Could you please help me with that?

For BlueSpice we have created REL1_31_dev branches in all repos, that we use to prepare the upcoming minor release. Could it be connected to that non-standard branch?

Event Timeline

Osnard created this task.May 24 2019, 12:35 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 24 2019, 12:35 PM
hashar added a subscriber: hashar.May 28 2019, 11:52 AM

I am not sure that is the reason for this change, but the non BlueSpice repositories are using the master branch, not the REL1_31_dev branch which does not exist in non BlueSpice repos.

The reason for the issue can be seen at the top of the job output with the lines prefixed with zuul.Cloner. That is the system that clone repositories, fetch and checkout code. Let me break it down there:

quibble.cmd:ZUUL_PROJECT=mediawiki/extensions/BlueSpiceVisualEditorConnector

The build has been triggered for the extension mediawiki/extensions/BlueSpiceVisualEditorConnector. The ZUUL_PROJECT environment variable is injected by CI automatically, as well as other values such as ZUUL_BRANCH=REL1_31_dev.

INFO:quibble.cmd:Projects: mediawiki/core, mediawiki/skins/Vector, mediawiki/extensions/BlueSpiceVisualEditorConnector, mediawiki/extensions/BlueSpiceFoundation, mediawiki/extensions/Cite, mediawiki/extensions/ExtJSBase, mediawiki/extensions/TemplateData, mediawiki/extensions/VisualEditor

The list of repositories that will be cloned. They have been injected via the environment variable EXT_DEPENDENCIES which is set by zuul/parameter_functions.py. All those repositories will be cloned. Then for each repository, the zuul.Cloner system will try in order to fetch and checkout:

  • A reference to the commit that CI prepared for the repository (refs/zuul/REL1_31_dev/Z862c67646e834967ad0889ff55b0d04d). That is because CI craft a merge commit of your patch against the tip of the branch (think of it like automatically rebasing before testing).
  • fall back to fetch the targeted branch: REL1_31_dev
  • fall back to master (which the last chance and is hardcoded in CI)

Quibble starts with mediawiki/core:

 INFO:quibble.zuul.clone:Cloning mediawiki/core first
 INFO:zuul.Cloner:Creating repo mediawiki/core from cache /srv/git/mediawiki/core.git
 INFO:zuul.Cloner:Updating origin remote in repo mediawiki/core to https://gerrit.wikimedia.org/r/mediawiki/core
 INFO:zuul.Cloner:upstream repo is missing branch REL1_31_dev
INFO:zuul.Cloner:Falling back to branch master

So since mediawiki/core does not have a REL1_31_dev branch, CI fall back to master and that is the same for a few other repositories (Vector, Cite, TemplateData etc) which all fall back to use the master branch instead.

Also the build attached the LocalSettings.php, its tail is:

# Enabled skins.
# The following skins were automatically enabled:
wfLoadSkin( 'Vector' );


# Enabled extensions. Most of the extensions are enabled by adding
# wfLoadExtensions('ExtensionName');
# to LocalSettings.php. Check specific extension documentation for more details.
# The following extensions were automatically enabled:
wfLoadExtension( 'BlueSpiceVisualEditorConnector' );
wfLoadExtension( 'Cite' );
wfLoadExtension( 'ExtJSBase' );
wfLoadExtension( 'TemplateData' );
wfLoadExtension( 'VisualEditor' );


# End of automatically generated settings.
# Add more configuration options below.

So it seems that BlueSpiceFoundation has not been injected in LocalSettings.php. Maybe because its extension.json expresses dependencies which are not fulfilled. In this case the installer just ignore the extension which is T220514 :-( Namely BlueSpiceFoundation REL1_31_dev has:

extension.json
"requires": {
    "MediaWiki": ">= 1.31.0",
    "extensions": {
        "ExtJSBase": "== 1.31"
    }
},

But as seen above, CI fetches ExtJSBase from the master branch which is most probably a different version and thus does not match == 1.31. And thus due to T220514, BlueSpiceFoundation is not loaded and the BlueSpice\Extensions class is not loaded.

Summary

  • BlueSpiceFoundation is not loaded
  • Repositories not having REL1_31_dev branches use master instead.

I would recommend to just use REL1_31 directly, then use tags when releasing a new version of the BlueSpice extensions. But I guess you have a different use case that ends up requiring a different branch. Will have to work out a solution of some sort for that use case.

Recently, requirement for ExtJSBase in BlueSpiceFoundation@REL1_31_dev was changed to "~3.1" which should match version "1.34" of ExtJSBase@master, but the tests still fail with the same error

Thanks for your explanations @hashar . I think we now have figured out how to do it. We set the version of ExtJSBase to be 1.31 in master. By this the CLI installer can add the wfLoadExtension call for "BlueSpiceFoundation" into LocalSettings.php even if it fetched ExtJSBase from the master branch.

Unfortunately we need to keep those non-standard branches as we can not develop new features in REL1_31 (as it is source of our rolling releases) but also not in master (as we can not target MediaWiki core at master). It's a mess. But I think we can live with it. Thanks for your support!

Osnard closed this task as Resolved.Jun 13 2019, 11:21 AM