@hashar We are now getting to the point where we want to test endpoints with different settings. For example, @nnikkhoui just wrote a REST endpoint for search and wants to test it with different search engines like CirrusSearch. Would the above solution allow us to test those different settings?

Ideally, I'd want to be able to keep the tests together and maybe tag[1] them with the type of search engine and then tell the CI to load the settings for certain tags and run them. Is there a way to do this?

[1]: https://github.com/mochajs/mocha/wiki/Tagging

local_settings.php is mostly tied to Quibble itself. Setting up environment variables (MW_INSTALL...), loading ./includes/DevelopmentSettings.php, $wgEnableJavaScriptTest = true.

In the recent past we had to:

• Enable the REST API /rest.php which is disabled by default: $wgEnableRestAPI = true T235564

Override settings generated by the installer:

• Have a fixed value for $wgSecretKey T230340

• Enable Upload for an integration test of the FileImporter extension $wgEnableUploads = true T190829

And indeed anytime we need a non default setting that currently means sending a patch to Quibble, cutting a new release, rebuilding the CI Docker containers. That is not sustainable at all.

If I understand the problem statement, the aim would be to have:

• MediaWiki setup with CirrusSearch and Elastica
• the api-testing tool
• an elasticsearch backend

Quibble already knows how to fetch the repositories and install them. We already use Quibble to clone and run the api-testing tool. Those are quite trivial.

Which leave us with:

1. have ElasticSearch and boot it up, possibly Quibble only knows about spawning the database (mysql/postgre) or the ChromeDriver service used for Selenium tests.
2. inject configuration to have MediaWiki to know about the elasticsearch ip/port.

@kostajh has a similar need to write an integration test using Parsoid (T218534). I thought it would be better achieved with the local-charts project which aims at starting a float of services and generating the related configuration settings for MediaWiki (@brennen and @jeena will correct me if I am wrong). That seems a better effort long term.

Meanwhile we might able to hack a job that use Quibble to do the setup, have a Docker container that has elasticsearch and boot it then somehow find a way to append the config at the end of LocalSettings.php. But that is really going to be dirty :-\

Maybe we can have the MediaWiki installer to recognize files in a localsettings.d?

My proposal for handling these types of services is to offload the work to docker-compose and let quibble interact with that stack. See T238224#5723037

Maybe we can have the MediaWiki installer to recognize files in a localsettings.d?

I have something like that in a proof of concept patch for T238224. It should probably be done in a separate patch, as well as the ability to override settings with environment variables which would make things easier too.