install.php --with-extensions silently ignores extensions whose dependencies are not satisfied
Closed, ResolvedPublic
Actions

Description

Mentioned in T220514#5235708.

maintenance/install.php --with-extensions silently skip extensions that lack another extension dependency. It used to break with an exception. The result is that CI ends up not testing some extensions at all since they are not loaded.

That comes from https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/461280/

The root cause is Installer::findExtensionsByType() which just skip faulty extensions. A summary would be:

$status = $this->getExtensionInfo( $type, $directory, $file );
if ( $status->isOK() ) {
    $exts[$file] = $status->value;
}
return $exts;

Which gets its informations from Installer::getExtensionInfo:

if ( $isJson ) {
    $jsonStatus = $this->readExtension( $fullJsonFile );
    if ( !$jsonStatus->isOK() ) {
        return $jsonStatus;
    }
    $info += $jsonStatus->value;
}

Since $jsonStatus is not ok, the extension is not added :-(

Details

	Subject	Repo	Branch	Lines +/-
	Revert and fix "Revert "Modify -—with-extensions to throw extension dependency errors""	mediawiki/core	master	+49 -18
	Modify -—with-extensions to throw extension dependency errors	mediawiki/core	master	+35 -15

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T220199 Quibble: deprecate and phase out EXT_DEPENDENCIES / SKIN_DEPENDENCIES
		Resolved		hashar	T225512 install.php --with-extensions silently ignores extensions whose dependencies are not satisfied

Event Timeline

CCicalese_WMF created this task.Jun 11 2019, 1:05 PM

Restricted Application removed a project: Patch-For-Review. · View Herald TranscriptJun 11 2019, 1:05 PM

CCicalese_WMF removed a parent task: T220514: MediaWiki web installer does not show extension when their dependency is missing.Jun 11 2019, 1:05 PM

CCicalese_WMF triaged this task as High priority.Jun 11 2019, 1:19 PM

CCicalese_WMF moved this task from Not a real column to Ready on the Platform Team Workboards board.

hashar updated the task description. (Show Details)Jun 24 2019, 2:09 PM

hashar added a parent task: T220199: Quibble: deprecate and phase out EXT_DEPENDENCIES / SKIN_DEPENDENCIES.Jun 24 2019, 4:03 PM

This missed the boat for the MW 1.33 release. Provisionally re-tagging to 1.34's release as well.

WDoranWMF edited projects, added Platform Team Workboards (Clinic Duty Team); removed Platform Team Workboards.Jul 5 2019, 3:11 PM

WDoranWMF lowered the priority of this task from High to Medium.Jul 17 2019, 8:35 PM

This is blocking RelEng, and they've called it out as a blocker in scrum-of-scrums. I'm changing the priority to high to get some focus, and I'd appreciate if this could get some attention in our next clinic duty to clear the blocker.

I'll schedule it into the next Clinic Duty Sprint

WDoranWMF moved this task from Inbox to Ready (WIP:5) on the Platform Team Workboards (Clinic Duty Team) board.Aug 13 2019, 6:40 PM

WDoranWMF removed a project: Platform Engineering (Needs Cleaning - Security, stability, performance, and scalability (TEC1)).Aug 13 2019, 7:04 PM

@WDoranWMF did this get assigned for the current Clinic Duty sprint? It looks like it's still in the "Ready" column.

It did unfortunately we're pretty low on MW resources this week :( I'll do my best.

Thanks. I want to go into scrum-of-scrums next week with head held high. The 1.34 release is wrapping up, and it'd be cool to catch this bug before it goes out.

Change 533239 had a related patch set uploaded (by Clarakosi; owner: Clarakosi):
[mediawiki/core@master] Modify -—with-extensions to throw extension dependency errors

https://gerrit.wikimedia.org/r/533239

gerritbot added a project: Patch-For-Review.Aug 29 2019, 4:19 PM

• Clarakosi moved this task from Ready (WIP:5) to Waiting for Review on the Platform Team Workboards (Clinic Duty Team) board.Aug 29 2019, 7:26 PM

• mobrovac moved this task from Waiting for Review to Ready (WIP:5) on the Platform Team Workboards (Clinic Duty Team) board.Sep 2 2019, 11:17 AM

Change 533239 merged by jenkins-bot:
[mediawiki/core@master] Modify -—with-extensions to throw extension dependency errors

https://gerrit.wikimedia.org/r/533239

• mobrovac closed this task as Resolved.Sep 3 2019, 12:51 PM

• mobrovac assigned this task to • Clarakosi.

• mobrovac removed a project: Patch-For-Review.

• mobrovac moved this task from Ready (WIP:5) to Done on the Platform Team Workboards (Clinic Duty Team) board.

In T225512#5298721, @Jdforrester-WMF wrote:

This missed the boat for the MW 1.33 release. Provisionally re-tagging to 1.34's release as well.

Do we want to backport for the 1.33 point release?

ReleaseTaggerBot added a project: MW-1.34-notes (1.34.0-wmf.21; 2019-09-03).Sep 3 2019, 1:00 PM

Reopening since merging this broke CI

So the mediawiki/core patch https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/533239/ is right and all fine as is. It causes the installer to fail early on when some dependencies are present which fulfill this task and I believe is the behavior we had before.

However, lot of repositories since to now express their dependencies in extension.json but CI does not use them yet. Instead the dependencies are centrally managed and injected to the CI job.

A detailed explanation with an example. The mediawiki/extensions/Graph repository extension.json does not define any requirements. However in CI we inject dependencies which can be seen in a build as the EXT_DEPENDENCIES parameter. For Graph that turns out to:

mediawiki/extensions/Cite
mediawiki/extensions/CodeEditor
mediawiki/extensions/ConfirmEdit
mediawiki/extensions/EventLogging
mediawiki/extensions/FlaggedRevs
mediawiki/extensions/JsonConfig
mediawiki/extensions/Scribunto
mediawiki/extensions/SyntaxHighlight_GeSHi
mediawiki/extensions/TemplateData
mediawiki/extensions/VisualEditor
mediawiki/extensions/WikiEditor
mediawiki/extensions/WikimediaEvents

So CI (via Quibble) clones that list of repositories and then run maintenance/install.php --with-extensions. With https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/533239/ , the installer process all the dependencies and fails since some repositories have not been found:

00:00:53.645 <ul><li>Could not find the registration file for the extension "Popups"</li>
00:00:53.645 <li>Could not find the registration file for the extension "Insider"</li>
00:00:53.646 <li>Could not find the registration file for the extension "Renameuser"</li>
00:00:53.646 <li>Could not find the registration file for the extension "TwoColConflict"</li>
00:00:53.646 <li>Could not find the registration file for the extension "Echo"</li>
00:00:53.647 <li>Could not find the registration file for the extension "WikibaseLexemeCirrusSearch"</li>
00:00:53.647 <li>Could not find the registration file for the extension "Babel"</li>
00:00:53.647 <li>Could not find the registration file for the extension "ArticlePlaceholder"</li>
00:00:53.648 <li>Could not find the registration file for the extension "WikibaseLexeme"</li>
00:00:53.648 <li>Could not find the registration file for the extension "PropertySuggester"</li>
00:00:53.648 <li>Could not find the registration file for the extension "DynamicSidebar"</li>
00:00:53.649 <li>Could not find the registration file for the extension "PageAssessments"</li>
00:00:53.649 <li>Could not find the registration file for the extension "LabeledSectionTransclusion"</li>
00:00:53.649 <li>Could not find the registration file for the extension "OATHAuth"</li>
00:00:53.650 <li>Could not find the registration file for the extension "Kartographer"</li>
00:00:53.650 <li>Could not find the registration file for the extension "ProofreadPage"</li>
00:00:53.651 <li>Could not find the registration file for the extension "Wikibase"</li>
00:00:53.651 <li>Could not find the registration file for the extension "Capiunto"</li>
00:00:53.651 <li>Could not find the registration file for the extension "CommonsMetadata"</li>
00:00:53.652 <li>Could not find the registration file for the extension "CiteThisPage"</li>
00:00:53.652 <li>Could not find the registration file for the extension "MassMessage"</li>
00:00:53.652 <li>Could not find the registration file for the extension "PageViewInfo"</li>
00:00:53.653 <li>Could not find the registration file for the extension "TranslationNotifications"</li>
00:00:53.653 <li>Could not find the registration file for the extension "CollaborationKit"</li>
00:00:53.653 <li>Could not find the registration file for the extension "Cognate"</li>
00:00:53.654 <li>Could not find the registration file for the extension "Math"</li>
00:00:53.654 <li>Could not find the registration file for the extension "PerformanceInspector"</li>
00:00:53.654 <li>Could not find the registration file for the extension "Quiz"</li>
00:00:53.655 <li>Could not find the registration file for the extension "WikibaseQualityConstraints"</li>
00:00:53.655 <li>Could not find the registration file for the extension "Citoid"</li>
00:00:53.655 <li>Could not find the registration file for the extension "timeline"</li>
00:00:53.656 <li>Could not find the registration file for the extension "PageImages"</li>
00:00:53.656 <li>Could not find the registration file for the extension "Jade"</li>
00:00:53.656 <li>Could not find the registration file for the extension "PagedTiffHandler"</li>
00:00:53.657 <li>Could not find the registration file for the extension "NavigationTiming"</li>
00:00:53.657 <li>Could not find the registration file for the extension "AntiSpoof"</li>
00:00:53.657 <li>Could not find the registration file for the extension "LocalisationUpdate"</li>
00:00:53.658 <li>Could not find the registration file for the extension "Flow"</li>
00:00:53.658 <li>Could not find the registration file for the extension "Score"</li>
00:00:53.659 <li>Could not find the registration file for the extension "GlobalBlocking"</li>
00:00:53.659 <li>Could not find the registration file for the extension "TorBlock"</li>
00:00:53.659 <li>Could not find the registration file for the extension "Newsletter"</li>
00:00:53.660 <li>Could not find the registration file for the extension "Sentry"</li>
00:00:53.660 <li>Could not find the registration file for the extension "Poem"</li>
00:00:53.660 <li>Could not find the registration file for the extension "Thanks"</li>
00:00:53.660 <li>Could not find the registration file for the extension "AbuseFilter"</li>
00:00:53.661 <li>Could not find the registration file for the extension "MachineVision"</li>
00:00:53.661 <li>Could not find the registration file for the extension "TrustedXFF"</li>
00:00:53.661 <li>Could not find the registration file for the extension "PageTriage"</li>
00:00:53.662 <li>Could not find the registration file for the extension "Interwiki"</li>
00:00:53.662 <li>Could not find the registration file for the extension "DoubleWiki"</li>
00:00:53.662 <li>Could not find the registration file for the extension "3D"</li>
00:00:53.663 <li>Could not find the registration file for the extension "RelatedArticles"</li>
00:00:53.663 <li>Could not find the registration file for the extension "VipsScaler"</li>
00:00:53.663 <li>Could not find the registration file for the extension "InputBox"</li>
00:00:53.664 <li>Could not find the registration file for the extension "NewUserMessage"</li>
00:00:53.664 <li>Could not find the registration file for the extension "FileImporter"</li>
00:00:53.664 <li>Could not find the registration file for the extension "UploadsLink"</li>
00:00:53.665 <li>Could not find the registration file for the extension "PoolCounter"</li>
00:00:53.665 <li>Could not find the registration file for the extension "CentralNotice"</li>
00:00:53.665 <li>Could not find the registration file for the extension "Collection"</li>
00:00:53.666 <li>Could not find the registration file for the extension "TemplateSandbox"</li>
00:00:53.666 <li>Could not find the registration file for the extension "CategoryTree"</li>
00:00:53.666 <li>Could not find the registration file for the extension "CongressLookup"</li>
00:00:53.666 <li>Could not find the registration file for the extension "MobileFrontend"</li>
00:00:53.667 <li>Could not find the registration file for the extension "ParsoidBatchAPI"</li>
00:00:53.667 <li>Could not find the registration file for the extension "GrowthExperiments"</li>
00:00:53.667 <li>Could not find the registration file for the extension "LoginNotify"</li>
00:00:53.668 <li>Could not find the registration file for the extension "RevisionSlider"</li>
00:00:53.668 <li>Could not find the registration file for the extension "EntitySchema"</li>
00:00:53.668 <li>Could not find the registration file for the extension "ReadingLists"</li>
00:00:53.669 <li>Could not find the registration file for the extension "ExtensionDistributor"</li>
00:00:53.669 <li>Could not find the registration file for the extension "MobileApp"</li>
00:00:53.669 <li>Could not find the registration file for the extension "QuickSurveys"</li>
00:00:53.670 <li>Could not find the registration file for the extension "Translate"</li>
00:00:53.670 <li>Could not find the registration file for the extension "GeoCrumbs"</li>
00:00:53.670 <li>Could not find the registration file for the extension "FileExporter"</li>
00:00:53.671 <li>Could not find the registration file for the extension "FeaturedFeeds"</li>
00:00:53.671 <li>Could not find the registration file for the extension "LdapAuthentication"</li>
00:00:53.671 <li>Could not find the registration file for the extension "Nuke"</li>
00:00:53.672 <li>Could not find the registration file for the extension "CirrusSearch"</li>
00:00:53.672 <li>Could not find the registration file for the extension "CodeReview"</li>
00:00:53.672 <li>Could not find the registration file for the extension "cldr"</li>
00:00:53.673 <li>Could not find the registration file for the extension "WikimediaMessages"</li>
00:00:53.673 <li>Could not find the registration file for the extension "CodeMirror"</li>
00:00:53.673 <li>Could not find the registration file for the extension "SearchExtraNS"</li>
00:00:53.673 <li>Could not find the registration file for the extension "InterwikiSorting"</li>
00:00:53.674 <li>Could not find the registration file for the extension "SecureLinkFixer"</li>
00:00:53.674 <li>Could not find the registration file for the extension "WikimediaMaintenance"</li>
00:00:53.674 <li>Could not find the registration file for the extension "AdvancedSearch"</li>
00:00:53.675 <li>Could not find the registration file for the extension "Calendar"</li>
00:00:53.675 <li>Could not find the registration file for the extension "GlobalPreferences"</li>
00:00:53.675 <li>Could not find the registration file for the extension "GeoData"</li>
00:00:53.676 <li>Could not find the registration file for the extension "GlobalCssJs"</li>
00:00:53.676 <li>Could not find the registration file for the extension "DismissableSiteNotice"</li>
00:00:53.676 <li>Could not find the registration file for the extension "TemplateWizard"</li>
00:00:53.677 <li>Could not find the registration file for the extension "ExternalGuidance"</li>
00:00:53.677 <li>Could not find the registration file for the extension "Elastica"</li>
00:00:53.677 <li>Could not find the registration file for the extension "DonationInterface"</li>
00:00:53.677 <li>Could not find the registration file for the extension "Linter"</li>
00:00:53.678 <li>Could not find the registration file for the extension "Dashiki"</li>
00:00:53.678 <li>Could not find the registration file for the extension "Petition"</li>
00:00:53.678 <li>Could not find the registration file for the extension "UrlShortener"</li>
00:00:53.679 <li>Could not find the registration file for the extension "OAuth"</li>
00:00:53.679 <li>Could not find the registration file for the extension "MultimediaViewer"</li>
00:00:53.679 <li>Could not find the registration file for the extension "FundraisingTranslateWorkflow"</li>
00:00:53.680 <li>Could not find the registration file for the extension "SubpageSortkey"</li>
00:00:53.680 <li>Could not find the registration file for the extension "SpamBlacklist"</li>
00:00:53.680 <li>Could not find the registration file for the extension "UniversalLanguageSelector"</li>
00:00:53.681 <li>Could not find the registration file for the extension "ContactPage"</li>
00:00:53.681 <li>Could not find the registration file for the extension "GlobalUsage"</li>
00:00:53.682 <li>Could not find the registration file for the extension "Josa"</li>
00:00:53.682 <li>Could not find the registration file for the extension "ArticleCreationWorkflow"</li>
00:00:53.682 <li>Could not find the registration file for the extension "ElectronPdfService"</li>
00:00:53.683 <li>Could not find the registration file for the extension "RSS"</li>
00:00:53.683 <li>Could not find the registration file for the extension "ORES"</li>
00:00:53.683 <li>Could not find the registration file for the extension "XAnalytics"</li>
00:00:53.684 <li>Could not find the registration file for the extension "BetaFeatures"</li>
00:00:53.684 <li>Could not find the registration file for the extension "Disambiguator"</li>
00:00:53.684 <li>Could not find the registration file for the extension "TextExtracts"</li>
00:00:53.685 <li>Could not find the registration file for the extension "GuidedTour"</li>
00:00:53.685 <li>Could not find the registration file for the extension "TimedMediaHandler"</li>
00:00:53.685 <li>Could not find the registration file for the extension "ApiFeatureUsage"</li>
00:00:53.686 <li>Could not find the registration file for the extension "LandingCheck"</li>
00:00:53.686 <li>Could not find the registration file for the extension "TocTree"</li>
00:00:53.686 <li>Could not find the registration file for the extension "OpenStackManager"</li>
00:00:53.687 <li>Could not find the registration file for the extension "GoogleNewsSitemap"</li>
00:00:53.687 <li>Could not find the registration file for the extension "SandboxLink"</li>
00:00:53.688 <li>Could not find the registration file for the extension "WikiLove"</li>
00:00:53.688 <li>Could not find the registration file for the extension "ContentTranslation"</li>
00:00:53.688 <li>Could not find the registration file for the extension "SiteMatrix"</li>
00:00:53.689 <li>Could not find the registration file for the extension "FundraiserLandingPage"</li>
00:00:53.689 <li>Could not find the registration file for the extension "ImageMap"</li>
00:00:53.689 <li>Could not find the registration file for the extension "wikihiero"</li>
00:00:53.690 <li>Could not find the registration file for the extension "GlobalUserPage"</li>
00:00:53.690 <li>Could not find the registration file for the extension "EventBus"</li>
00:00:53.690 <li>Could not find the registration file for the extension "ActiveAbstract"</li>
00:00:53.691 <li>Could not find the registration file for the extension "CharInsert"</li>
00:00:53.691 <li>Could not find the registration file for the extension "PdfHandler"</li>
00:00:53.691 <li>Could not find the registration file for the extension "intersection"</li>
00:00:53.692 <li>Could not find the registration file for the extension "SkinPerPage"</li>
00:00:53.692 <li>Could not find the registration file for the extension "TemplateStyles"</li>
00:00:53.692 <li>Could not find the registration file for the extension "Listings"</li>
00:00:53.693 <li>Could not find the registration file for the extension "GettingStarted"</li>
00:00:53.693 <li>Could not find the registration file for the extension "GWToolset"</li>
00:00:53.693 <li>Could not find the registration file for the extension "SecurePoll"</li>
00:00:53.694 <li>Could not find the registration file for the extension "CreditsSource"</li>
00:00:53.694 <li>Could not find the registration file for the extension "ShortUrl"</li>
00:00:53.694 <li>Could not find the registration file for the extension "MapSources"</li>
00:00:53.695 <li>Could not find the registration file for the extension "WikimediaBadges"</li>
00:00:53.695 <li>Could not find the registration file for the extension "Wikidata.org"</li>
00:00:53.695 <li>Could not find the registration file for the extension "ContributionTracking"</li>
00:00:53.696 <li>Could not find the registration file for the extension "EUCopyrightCampaign"</li>
00:00:53.696 <li>Could not find the registration file for the extension "CheckUser"</li>
00:00:53.696 <li>Could not find the registration file for the extension "LiquidThreads"</li>
00:00:53.697 <li>Could not find the registration file for the extension "WikidataPageBanner"</li>
00:00:53.697 <li>Could not find the registration file for the extension "CentralAuth"</li>
00:00:53.697 <li>Could not find the registration file for the extension "TitleBlacklist"</li>
00:00:53.698 <li>Could not find the registration file for the extension "WikibaseMediaInfo"</li>
00:00:53.698 <li>Could not find the registration file for the extension "UserMerge"</li>
00:00:53.698 <li>Could not find the registration file for the extension "WikimediaEditorTasks"</li>
00:00:53.699 <li>Could not find the registration file for the extension "TheWikipediaLibrary"</li>
00:00:53.699 <li>Could not find the registration file for the extension "WikibaseCirrusSearch"</li>
00:00:53.699 <li>Could not find the registration file for the extension "BounceHandler"</li>
00:00:53.700 <li>Could not find the registration file for the extension "Campaigns"</li>
00:00:53.700 <li>Could not find the registration file for the extension "Gadgets"</li>
00:00:53.700 <li>Could not find the registration file for the extension "WikimediaIncubator"</li>
00:00:53.701 <li>Could not find the registration file for the extension "UploadWizard"</li>
00:00:53.701 <li>Could not find the registration file for the extension "ParserFunctions"</li>
00:00:53.701 <li>Could not find the registration file for the extension "SubPageList3"</li></ul>

Which is pretty much the whole planet. Yeah dependencies are hard :-\ I guess because requirements have been added and since the installer was no more breaking, they kept being added unnoticed to CI.

We need to synchronize the merge of that mediawiki/core change with a change to CI to stop injecting the dependencies and instruct Quibble to clone the dependencies listed in extension.json. Or in short:

Make wmf-quibble to have all the dependencies. I think I have a test that covers what is missing
Change wmf-quibble to use quibble --resolve-requires --fail-on-extra-requires. To ensure the wmf-quibble jobs has the whole dependency tree
Adjust Quibble jobs to use quibble --resolve-requires
Remove the logic in CI that injects EXT_DEPENDENCIES
Migrate those dependencies to each of the extension.json / skin.json
Merge the mediawiki/core change

And that would more or less work.

Sorry I have short sighted the effect of having the mediawiki/core change merged (and again the change does exactly what I have requested).

Perhaps the patch could be changed to warn about extensions being skipped, instead of failing if any extensions have missing dependencies. Always, or gated by a --force-extensions flag.

WDoranWMF moved this task from Inbox to Blocked Externally on the Platform Team Workboards (Clinic Duty Team) board.Sep 4 2019, 1:44 PM

@hashar I'd like to keep momentum on this so we can try to close it out, would Daniel's suggestion work for the behaviour you need or what are your preferred next steps?

TLDR the installer should ignore/skip extension and skins that do not have an extension.json / skin.json or the legacy PHP entry point (eg: extensions/Foobar/Foobar.php).

I would like to revisit the mediawiki/core patch and figure out what craziness ends up listing 170 extensions as missing when one has sends a patch to mediawiki/extensions/Graph with the CI dependencies:

The list shows CongressLookup or FundraisingTranslateWorkflow, but neither of them are listed in the dependencies injected by CI nor are they listed in any extension.json. The reason they show up is an entire mystery to me.

And *magic* I found the reason just now as I was writing my comment ;-]

The failure and huge list comes from a change to Graph made to the wmf branch https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Graph/+/534188/ . The mediawiki/core wmf branch has 183 extensions registered as submodules, the CI build has cloned 13 of them, but the 170 other are listed as submodules and ARE not process, they are thus empty directories.

The errors that are shown is emitted due to a lack of extension.json, that case should be ignored. It was ignored previously. So I guess we can tweak the patch to not fail when the extension is empty. The code before 5e9ada58821e7e54fed9fed5dd2b1e7968967067 did:

git show 5e9ada58821e7e54fed9fed5dd2b1e7968967067^:includes/installer/Installer.php

1285         // extensions -> extension.json, skins -> skin.json
1286         $jsonFile = substr( $directory, 0, strlen( $directory ) - 1 ) . '.json';    # path to extension.json / skin.json
1287 
1288         $dh = opendir( $extDir );
1289         $exts = [];
1290         while ( ( $file = readdir( $dh ) ) !== false ) {
1291             if ( !is_dir( "$extDir/$file" ) ) {
1292                 continue;
1293             }
1294             $fullJsonFile = "$extDir/$file/$jsonFile";
1295             $isJson = file_exists( $fullJsonFile );  # Whether the extension.json/skin.json file exists
1296             $isPhp = false;
1297             if ( !$isJson ) {
1298                 // Only fallback to PHP file if JSON doesn't exist
1299                 $fullPhpFile = "$extDir/$file/$file.php";
1300                 $isPhp = file_exists( $fullPhpFile );
1301             }
1302             if ( $isJson || $isPhp ) {   # else do not bother and skip
1303                 // Extension exists. Now see if there are screenshots
1304                 $exts[$file] = [];
1305                 if ( is_dir( "$extDir/$file/screenshots" ) ) {
1306                     $paths = glob( "$extDir/$file/screenshots/*.png" );
1307                     foreach ( $paths as $path ) {
1308                         $exts[$file]['screenshots'][] = str_replace( $extDir, "../$directory", $path );
1309                     }
1310 
1311                 }
1312             }
1313             if ( $isJson ) {
1314                 $info = $this->readExtension( $fullJsonFile );
1315                 if ( $info === false ) {
1316                     continue;
1317                 }
1318                 $exts[$file] += $info;
1319             }
1320         }

So essentially, the old logic would just skip extension or skin directories that do not have extension.json or skin.json (respectively). So I guess that is the part missing from the mediawiki/core

Other things:

the mediawiki/core wmf branch also has skins as submodules, but they are not listed as missing the registration file
the message uses HTML formatting for the CLI output :-]

@WDoranWMF asked me to fix up the reverted patch.

In T225512#5468074, @hashar wrote:

TLDR the installer should ignore/skip extension and skins that do not have an extension.json / skin.json or the legacy PHP entry point (eg: extensions/Foobar/Foobar.php).

To confirm, that's the only issue? The concern over CI not processing dependencies in T225512#5462372 is actually not an issue because the dependencies are manually included in the CI config?

Other things:

the mediawiki/core wmf branch also has skins as submodules, but they are not listed as missing the registration file

It checks extensions and skins separately. So it errors out on the missing extensions before it gets a chance to check skins.

the message uses HTML formatting for the CLI output :-]

Fixed that too.

Change 534641 had a related patch set uploaded (by Anomie; owner: Anomie):
[mediawiki/core@master] Revert and fix "Revert "Modify -—with-extensions to throw extension dependency errors""

https://gerrit.wikimedia.org/r/534641

gerritbot added a project: Patch-For-Review.Sep 5 2019, 4:51 PM

In T225512#5468701, @Anomie wrote:

@WDoranWMF asked me to fix up the reverted patch.

In T225512#5468074, @hashar wrote:

TLDR the installer should ignore/skip extension and skins that do not have an extension.json / skin.json or the legacy PHP entry point (eg: extensions/Foobar/Foobar.php).

To confirm, that's the only issue? The concern over CI not processing dependencies in T225512#5462372 is actually not an issue because the dependencies are manually included in the CI config?

Have to do more test. I discovered the issue when an extension did not get included because one of its requirements in extension.json was not available. Eg WikibaseMediaInfo would not be loaded if you lack one of UniversalLanguageSelector or WikibaseCirrusSearch. That is the main intent for this patch, die out instead of silently ignoring an extension which lack some requires.

Once merged, I guess some repositories/branches will break here due to CI not injecting dependencies and thus not fulfilling the requirement in extension.json. Those would be easy to fix (just add them back to integration/config), and I might even verify it ahead of time by comparing what is defined in all the extension.json and amend integration/config to match.

I will write down a test plan of the few combinations and see how the mediawiki/core patch behave. The train went fine this week luckily so I got plenty of time this Friday :]

The next thing is to stop injecting dependencies from CI and instead have the test runner (Quibble) to process extension.json requires field and recursively clone the extensions then run the installer. So that developers can freely add requirements as may fit without having to use integration/config.git. The testrunner has all the logic which can be enabled via quibble --resolve-requires [--fail-on-extra-requires], have to think of a plan to smoothly migrate extensions one by one to that new system. That last part is on my side ;-]

Oh, we specifically want it to error out on missing dependencies. Thanks for the clarification.

Yes sorry my request was not very clear in the first place. Seems like I originally filled T220514 MediaWiki web installer do not show extension when their dependency is missing which mentions the issue about wfLoadExtension( 'WikibaseMediaInfo' );. not being injected due to one of its requirement not being available, which is annoying when the patch is for WikibaseMediaInfo :]

Eventually I rephrased it as "Webinstaller do not show extension missing dependencies" which is related but confused people as the task being to fix the web installer when I really wanted to have the CLI to fail. So in the end nobody (including me) would knew what was the feature / fix being requested. I take the blame for confusing two different issues albeit related.

• Clarakosi removed • Clarakosi as the assignee of this task.Sep 5 2019, 8:25 PM

• Clarakosi subscribed.

Change 534641 merged by jenkins-bot:
[mediawiki/core@master] Revert and fix "Revert "Modify -—with-extensions to throw extension dependency errors""

https://gerrit.wikimedia.org/r/534641

Tentatively resolved :] Seems to behave properly so far.

thank you @Clarakosi @Anomie @mobrovac :)

WDoranWMF moved this task from Blocked Externally to Done on the Platform Team Workboards (Clinic Duty Team) board.Sep 9 2019, 8:32 PM

And I wrote a quick announce to wikitech-l about having to define the requirements in CI config in addition of extension.json: https://lists.wikimedia.org/pipermail/wikitech-l/2019-September/092522.html

ReleaseTaggerBot edited projects, added MW-1.34-notes (1.34.0-wmf.22; 2019-09-10); removed MW-1.34-notes (1.34.0-wmf.21; 2019-09-03).Sep 9 2019, 9:00 PM

I don't think this had the desired effect - From a patch to WikimediaMessages

https://integration.wikimedia.org/ci/job/mwext-php72-phan-docker/11177/console

22:56:02 A dependency error was encountered while installing the extension "MinervaNeue": Could not find the registration file for the extension &#34;MobileFrontend&#34;
22:56:02 [12387160f4db169a2121e2f1] [no req]   Wikimedia\Services\ServiceDisabledException from line 423 of /workspace/src/includes/libs/services/ServiceContainer.php: Service disabled: DBLoadBalancerFactory
22:56:02 Backtrace:
22:56:02 #0 /workspace/src/includes/MediaWikiServices.php(577): Wikimedia\Services\ServiceContainer->getService(string)
22:56:02 #1 /workspace/src/maintenance/doMaintenance.php(123): MediaWiki\MediaWikiServices->getDBLoadBalancerFactory()
22:56:02 #2 /workspace/src/maintenance/install.php(193): require_once(string)
22:56:02 #3 {main}
22:56:02 Traceback (most recent call last):
22:56:02   File "/usr/local/bin/quibble", line 11, in <module>
22:56:02     load_entry_point('quibble==0.0.0', 'console_scripts', 'quibble')()
22:56:02   File "/usr/local/lib/python3.5/dist-packages/quibble/cmd.py", line 465, in main
22:56:02     cmd.execute(plan)
22:56:02   File "/usr/local/lib/python3.5/dist-packages/quibble/cmd.py", line 440, in execute
22:56:02     command.execute()
22:56:02   File "/usr/local/lib/python3.5/dist-packages/quibble/commands.py", line 428, in execute
22:56:02     mwdir=self.mw_install_path
22:56:02   File "/usr/local/lib/python3.5/dist-packages/quibble/mediawiki/maintenance.py", line 63, in install
22:56:02     'Install failed with exit code: %s' % p.returncode)

About "Service disabled" warning, see T229601.

Krinkle reopened this task as Open.Sep 10 2019, 12:47 AM

Krinkle merged a task: T232413: mwext-php72-phan-docker job fails on WikimediaMessages: A dependency error was encountered while installing the extension "MinervaNeue": Could not find the registration file for the extension "MobileFrontend".

Krinkle mentioned this in T232413: mwext-php72-phan-docker job fails on WikimediaMessages: A dependency error was encountered while installing the extension "MinervaNeue": Could not find the registration file for the extension "MobileFrontend".

Krinkle added a subscriber: Kizule.

Krinkle closed this task as Resolved.Sep 10 2019, 12:50 AM

CCicalese_WMF mentioned this in T220514: MediaWiki web installer does not show extension when their dependency is missing.Oct 1 2019, 2:43 PM

Aklapper removed a subscriber: Anomie.Oct 16 2020, 5:40 PM