Page MenuHomePhabricator

[Spike] Can we use the mediawiki-config repo for our Visual Regression test environment?
Closed, DeclinedPublic3 Estimated Story PointsSpike

Description

Ideally, our visual regression tests would simulate a production environment as much as possible so that when we make a change, we can be extremely confident that the change won't cause unintended side effects (especially of the visual kind) once it hits production. One aspect of simulating a production environment is having the same config (or nearly the same config) as production.

The mediawiki-config repo hosts the configuration for our production MediaWiki sites where it includes a number of important files:

See the README for more info that may be pertinent

Question we are trying to answer

Can we use mediawiki-config to power a MediaWiki app running inside a Docker container and, if so, how?

  • It looks like Ahmon Dancy has done great work with Docker and utilizing mediawiki-config in P16947 . Could we use this? Schedule a meeting and determine how feasible it is to use mediawiki-config to produce a production like environment for visual regression testing. If it's not feasible, is there a better strategy (e.g. Jon mentioned possibly applying a patch to the wikimedia-debug servers)?

Event Timeline

nray updated the task description. (Show Details)
nray set the point value for this task to 0.
Restricted Application changed the subtype of this task from "Deadline" to "Spike". · View Herald TranscriptApr 7 2022, 11:02 PM

@dancy I noticed your great work in setting up a docker multiversion mediawiki environment in https://phabricator.wikimedia.org/P16947 . We are looking in setting up a mediawiki docker container that uses mediawiki-config for visual regression testing. Would it be possible to use this image? I tried setting it up but ran into a certificate error

Hi @nray. It might be possible for you to use the multiversion mediawiki container for testing. It requires a "webserver" image to be used along with it. That can be created by running make -C make-container-image/webserver.

I assume you already know this but I will emphasize here that the multiversion image expects to be able to access production resources (databases, etc) when it is used.

Generally speaking I'm happy to work with you to hack something up that suits your needs.

Regarding the certificate error, can you supply the command you used for testing?

ovasileva changed the point value for this task from 0 to 3.Apr 12 2022, 5:38 PM
nray updated the task description. (Show Details)
nray updated the task description. (Show Details)

Thanks for your response @dancy . Regarding the cert error I was experiencing, I followed the instructions P16947 but had to modify some things to get around other some other issues. For example, I removed this line because there seemed to already be a wikiversion-dev.php in the image I produced from P16947. I was probably doing something wrong though and AFAIK I wasn't using the webserver image.

Thank you also for the notice about the production resources. Ideally, we'd want to be able to somehow modify the config to point to local resources (for example a mariadb database running in another container). Do you think that's feasible or would that require a lot of work?

The directions in P16947 are for creating an image that uses the train-dev branch of operations/mediawiki-config. This branch has a modified configuration that allows the image (along with some external components such as memcached, etcd, and mysql database) to work just enough to return a good response for en.wikipedia.org/wiki/Special:Version. A lot of that involved turning off extensions, etc to reduce the number of external requirements.

For your case you could probably do something similar (i.e, make a different branch of operations/mediawiki-config with a configuration modified to suit your needs). I will note that maintaining a separate branch can become a burden since you have to merge in changes from master from time to time (resolving conflicts and dealing with configuration drift in the process).

Note that I'm off work this week (Apr 18-22) so I won't be able to offer assistance until after that.

Thank you for all the the info @dancy

Our team talked about this ticket today and agreed that, at least for now, we are content with going with a more barebones mediawiki setup/config and seeing how far that takes us. As a result, I am declining this ticket