Change 587848 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: Add Kartographer to the gate

https://gerrit.wikimedia.org/r/587848

Change 587848 merged by jenkins-bot:
[integration/config@master] layout: Add Kartographer to the gate

https://gerrit.wikimedia.org/r/587848

Change 587858 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: Add WikimediaMessages to the gate

https://gerrit.wikimedia.org/r/587858

Change 587858 merged by jenkins-bot:
[integration/config@master] layout: Add WikimediaMessages to the gate

https://gerrit.wikimedia.org/r/587858

Change 596707 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] gate: Add BetaFeatures

https://gerrit.wikimedia.org/r/596707

Change 596707 merged by jenkins-bot:
[integration/config@master] gate: Add BetaFeatures

https://gerrit.wikimedia.org/r/596707

Mentioned in SAL (#wikimedia-releng) [2020-05-15T17:48:07Z] <James_F> Zuul: Add BetaFeatures to the gate T249674

For consideration for MW-on-K8s.

Change 655097 had a related patch set uploaded (by Jforrester; owner: C. Scott Ananian):
[integration/config@master] Zuul: Add Disambiguator to the MediaWiki gated extension set

https://gerrit.wikimedia.org/r/655097

Change 655097 merged by jenkins-bot:

[integration/config@master] Zuul: Add Disambiguator to the MediaWiki gated extension set

https://gerrit.wikimedia.org/r/655097

Mentioned in SAL (#wikimedia-releng) [2021-05-11T23:32:01Z] <James_F> Zuul: Add Disambiguator to the MediaWiki gated extension set T237538 T249674

Change 687769 had a related patch set uploaded (by Jforrester; author: Kosta Harlan):

[integration/config@master] Zuul: Add GrowthExperiments to the MediaWiki gated extension set

https://gerrit.wikimedia.org/r/687769

Change 687769 merged by jenkins-bot:

[integration/config@master] Zuul: Add GrowthExperiments to the MediaWiki gated extension set

https://gerrit.wikimedia.org/r/687769

Mentioned in SAL (#wikimedia-releng) [2021-05-12T16:00:48Z] <James_F> Zuul: Add GrowthExperiments to the MediaWiki gated extension set T247507 T249674

In T249674#7082525, @gerritbot wrote:

Change 687769 merged by jenkins-bot:

[integration/config@master] Zuul: Add GrowthExperiments to the MediaWiki gated extension set

https://gerrit.wikimedia.org/r/687769

It looks like there's a problem because SuggestedEditsTest.php assumes that PageViewInfo extension is present, but it's not in the gate. See e.g. https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-php72-docker/63679/console with stacktrace:

11:47:54 1) GrowthExperiments\Tests\HomepageModules\SuggestedEditsTest::testNoTopicFiltersWhenTopicMatchingDisabled
11:47:54 Error: Call to undefined method Mock_PageViewService_963a7339::supports()
11:47:54 
11:47:54 /workspace/src/extensions/GrowthExperiments/includes/HomepageModules/SuggestedEdits.php:701
11:47:54 /workspace/src/extensions/GrowthExperiments/includes/HomepageModules/SuggestedEdits.php:418
11:47:54 /workspace/src/extensions/GrowthExperiments/includes/HomepageModules/BaseModule.php:156
11:47:54 /workspace/src/extensions/GrowthExperiments/includes/HomepageModules/BaseModule.php:137
11:47:54 /workspace/src/extensions/GrowthExperiments/includes/HomepageModules/SuggestedEdits.php:264
11:47:54 /workspace/src/extensions/GrowthExperiments/includes/HomepageModules/BaseModule.php:97
11:47:54 /workspace/src/extensions/GrowthExperiments/tests/phpunit/unit/HomepageModules/SuggestedEditsTest.php:42
11:47:54 /workspace/src/tests/phpunit/MediaWikiUnitTestCase.php:108

PageImages is also not in gate but GrowthExperiments has that defined as a dependency in integration/config.
cc @DLynch

Change 690004 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/config@master] Zuul: Add PageViewInfo to gated extensions

https://gerrit.wikimedia.org/r/690004

Change 690028 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Zuul: Add PageImages to the MediaWiki gated extension set

https://gerrit.wikimedia.org/r/690028

Change 690004 abandoned by Kosta Harlan:

[integration/config@master] Zuul: Add PageViewInfo to gated extensions

Reason:

https://gerrit.wikimedia.org/r/690004

Change 690004 restored by Jforrester:

[integration/config@master] Zuul: Add PageViewInfo to gated extensions

https://gerrit.wikimedia.org/r/690004

Change 690028 merged by jenkins-bot:

[integration/config@master] Zuul: Add PageImages to the MediaWiki gated extension set

https://gerrit.wikimedia.org/r/690028

Change 690004 merged by jenkins-bot:

[integration/config@master] Zuul: Add PageViewInfo and Graph to gated extensions

https://gerrit.wikimedia.org/r/690004

Mentioned in SAL (#wikimedia-releng) [2021-05-12T19:32:06Z] <James_F> Zuul: Add PageImages, PageViewInfo, and Graph to MediaWiki gated extension set T249674

Change 939290 had a related patch set uploaded (by Majavah; author: Majavah):

[integration/config@master] zuul: [mediawiki/extensions/WikiLove] Add to gate

https://gerrit.wikimedia.org/r/939290

Change 939290 merged by jenkins-bot:

[integration/config@master] zuul: [mediawiki/extensions/WikiLove] Add to gate

https://gerrit.wikimedia.org/r/939290

Mentioned in SAL (#wikimedia-releng) [2023-07-20T07:17:01Z] <hashar> Reloaded Zuul to add WikiLove to gated extensions | https://gerrit.wikimedia.org/r/c/integration/config/+/939290 | T249674

TLDR: I am inclined to not pursue this idea and decline this task. Cross extensions tests can be covered by injecting each others as dependencies in CI which saves us from having to add more tests to the shared gate. Ultimately I think concept should probably disappear.

History

I have introduced Zuul as the CI test runner because it makes it possible to ensure that repositories do not break others. We had an RFC about it a decade ago T1350 and the first implementation is at https://gerrit.wikimedia.org/r/c/integration/config/+/180494 and only included a small subset of extensions:

Echo
JsonConfig
Mantle
MobileApp
MobileFrontend
VisualEditor
WikiGrok
ZeroBanner
ZeroPortal

More extensions have then been added to it over a decade and the aul was to eventually have all the Wikimedia deployed extensions participating in that shared job and thus ensure neither core or any extension will break another one.

Problems

That approaches was rather naive and has several issues:

• Wikis have different set of extensions as configured in operations/mediawiki-config and we do not take that in account
• Some extensions in the gate depends on extension which are not in it. A change can then enter one of its dependency which ends up breaking the gate for everyone. I had a dashboard to detect those and complement the list, but we had other priorities/reorgs etc
• Our testsuite / Quibble runs every single test it discovers. A decade ago it was good enough to blindly run a few hundred lightweight tests, nowadays we have:
  • an order of magnitude more tests (Wikibase has lot of tests for example)
  • some suites are quite heavy (Selenium tests).
• We do not even cover some critical parts of the infra such as CentralAuth T333541
• For sometime now, we have refrained from adding more extensions because that adds to the CI running time for each of the repositories participating (which includes a Selenium job taking more than 20 minutes).

Testing cross extensions coupling

T403560 is a request to add ReadingList to the gate and it has some good discussions on that topic. It discusses to have ReadingList tests running when changes are made to MinervaNeue and Vector. The easy path is to add ReadingList to the gate which would cause its CI to be slower due to all tests running and its test to run for every other extensions. Instead I have proposed to have ReadingList tested with both skins:

ReadingLists:
 - MinervaNeue
 - Vector

And add the reverse dependency as well:

MinervaNeue:
 - ReadingLists

Vector:
 - ReadingLists

Ensuring changes to dependencies do not break dependent repositories is the unaddressed T318107.

Then the task for ReadingList was due to a breaking change in mediawiki/core and that would not have caught it. Can we ensure changes to MediaWiki core do not break extensions? Yes surely, but we do not have a good/quick/speedy solution for that.

Conclusion

I would like to decline this task and not pursue it further. Blindly running every tests from all repositories does not scale at all and we will need a finer approach. We can devise a smarter system though (manually tagging tests or maybe automatically detecting which tests are affected across repositories depending on the repository being tested). There are some solution which I believe would be easier once we have upgraded Zuul (Continuous-Integration-Infrastructure (Zuul upgrade) ).

Cleanup, this is not MW-on-K8s related.