Specifically, it would be very nice if we could influence the MW docker-compose configuration when testing the Wikilambda extension. This way, we could introduce other services (viz. function-orchestrator and function-evaluator) for purposes of integration testing.
We do have a way to run integration tests via helm (deploying to kubernetes) in CI. I think once the mediawiki chart is ready, we could probably try to get something like this working. We'd need to make sure the staging cluster/CI namespace has enough resources to support it.
I don't know of any discussions or plans by RelEng to support docker-compose in CI, but my knowledge is limited to the Pipelinelib in that regard. Migration to gitlab could also affect all of this. @brennen for any questions about gitlab.
All of the above refers to automated testing. I'm also trying to get a preview environment project underway for the UAT/manual testing part of this, but don't have any phabricator tasks to point you to at the moment.
On the legacy CI infrastructure (Zuul / Jenkins / Docker images) the workload is run on WMCS instances that have a Docker daemon. We can not use that daemon to run arbitrary containers from a docker-compose definition. So what Jeena said: the target is to use Helm charts / Kubernetes much like we will do for production deployments.
I would be very interested in learning more about using helm/kubernetes to deploy a CI test environment (or whatever else you would suggest today, I realize this thread is quite old.) Are there anu docs or examples you could point me to? Is there anything that would be blocking me from trying it out?