I have a somewhat similar use case but very different in the sense that I'd like to have mw-config as a dep to my extension during unit tests.
I wrote code that depends on SiteMatrix, this extension depends heavily on mediawiki-config as it exposes very WMF specific ways of linking wikis to each others.
The current strategy I used today is to copy all the data from wmf-config to a resource folder in my unit tests:
It looks like this : https://gerrit.wikimedia.org/r/#/c/320757
As you can see a lot of resources needed to be copied from wmf-config.
Unfortunately this was the only way for me to develop code on top of SiteMatrix. But copying the resources helped me only for initial development, it won't allow CI to detect failures if "breaking changes" are added to wmf-config.

Change 320792 had a related patch set uploaded (by Hashar):
(WIP) clone wmf branches for operations/mediawiki-config

https://gerrit.wikimedia.org/r/320792

context

That came back up. The challenge at the time was to git clone mediawiki/core a couple time which typically takes a few minutes for a full clone. Nowadays the disposable instances have a local git mirroring repo to clone from which is dramatically faster (git uses hardlink).

The next thing is that the instances do NOT have phpunit installed. It has to be installed via composer. However mediawiki-config comes with the dependencies fetched in and there is no direct support to add extra dependencies / run composer from root of the repo. T85947 / https://gerrit.wikimedia.org/r/#/c/189148/ would do it if I understand it properly.

POC

The idea is to extract the current wmf branches from the wikiversions.json file. Then clone them as php-$version using the local mirror as reference (/srv/git/mediawiki/core.git).

Had to ship phpunit which is not possible right now (but will with be inclued with https://gerrit.wikimedia.org/r/#/c/189148/ . A simple composer require phpunit/phpunit=4.8.* does it.

From there, I have hit a wall with one of xhprof dependency requiring the mcrypt extension which we do not have. Fix is to have composer to ignore the platform requirements entirely. Simply pass to composer --ignore-platform-reqs.

Then it is all about invoking phpunit and publishing the test result.

The draft CI configuration change is: https://gerrit.wikimedia.org/r/320792

An example run:

• console
• test report

NEXT

I clearly remember that @EBernhardson had a use case to have mediawiki/core included. I guess that got worked around by copy pasting to operations/mediawiki-config. We can try having a patch that clear up the code duplication then run the job above against that patch.

For CirrusSearch, the submodules are not updated. That would require more than 300 clones which is going to take age (and the instances do not mirror all repos). But maybe we can have the job to solely submodule update CirrusSearch and run the subset of integration tests it would need.

My fear is that the whole thing might well be too complicated to maintain in the long run, but it is probably worth having operations/mediawiki-config patches to have some integration test run against them.

If one want to build the job with a operations/mediawiki-config patch:

https://integration.wikimedia.org/ci/job/operations-mw-config-phpunit-2/build (eventually login with your wmf labs account) then:

That should fetch and checkout the patch

Change 320980 had a related patch set uploaded (by DCausse):
[WIP] test job jenkins with mw-core

https://gerrit.wikimedia.org/r/320980

@hashar thanks!

I tested a patch where I removed SiteConfiguration but I don't know how to import it yet:
build output: https://integration.wikimedia.org/ci/job/operations-mw-config-phpunit-2/15/console
patch: https://gerrit.wikimedia.org/r/#/c/320980/

The PHPUnit job lacks the configuration to point to one or both the MediaWiki version that has been cloned. Else:

Fatal error:  Class 'SiteConfiguration' not found in /home/jenkins/workspace/operations-mw-config-phpunit-2/wmf-config/wgConf.php

+ https://gerrit.wikimedia.org/r/#/c/332777/1

Change 332777 had a related patch set uploaded (by Hashar):
(WIP) run tests against multiple mw versions (WIP)

https://gerrit.wikimedia.org/r/332777

Change 320980 abandoned by DCausse:
[WIP] test job jenkins with mw-core

Reason:
no :)

https://gerrit.wikimedia.org/r/320980

Change 332777 abandoned by Hashar:
(WIP) run tests against multiple mw versions (WIP)

https://gerrit.wikimedia.org/r/332777

Change 320792 abandoned by Hashar:
(WIP) clone wmf branches for operations/mediawiki-config

https://gerrit.wikimedia.org/r/320792

Sorry that will need to be revisited eventually :/