Change 537118 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[integration/quibble@master] Force value of $wgSecretKey.

https://gerrit.wikimedia.org/r/537118

What about using the RunSingleJob.php RPC end point that is used in production? It doesn't require a special token and since it's the way it's done in production, it brings CI tests closer to a more realistic set-up.

In T233092#5498618, @mobrovac wrote:

What about using the RunSingleJob.php RPC end point that is used in production? It doesn't require a special token and since it's the way it's done in production, it brings CI tests closer to a more realistic set-up.

SpeicalRunSingleJob does require the mediawiki_signature field, which is verified using $wgSecretKey. Same problem.

Also, RunSingleJob wouldn't work for this purpose, because it doesn't run jobs from the queue. It runs the job that was sent by the client in the request body. And it's not in core ;)

I'm not familiar with MediaWiki internals at all, so please excuse me for asking what may be excessive basic questions. Is the following a sufficiently correct understanding: CPT wants CI to deploy an MW test instance to a test machine, and want to run integration or other tests against that machine. The tests will require a shared secret between the test and the software running on the test machine.

If that's correct enough, I suggest it's CI's job to install the shared secret somewhere suitable on the test machine, but the job of MW developers to specify where.

It might be something Quibble can handle on its own, of course.

From a general security standpoint, I recommend against environment variables, as the values of those leak too easily. A configuration file with permissions locked down seems like a reasonable way for most things. I also recommend against having hard-coded, well-known secrets for test purposes, as those have a tendency to end up in production, and if they're sufficiently entrenched, it's hard to change them. Thus, having CI or Quibble generate a new, random, strong secret for each test run seems like a good way of avoiding bad habits.

In T233092#5540581, @LarsWirzenius wrote:

If that's correct enough, I suggest it's CI's job to install the shared secret somewhere suitable on the test machine, but the job of MW developers to specify where.

That's close, but my understanding is that we are talking about two docker containers (my understanding of CI infrastructure is rather fuzzy, @holger.knust probably knows better, @hashar definitly does). One is running MediaWiki, the other is running the test suite (Mocha on node.js).

MediaWiki expects the secret in LocalSettings.php, but we could pull it in from wherever. As long as it's in a known location on the file system or the environment, I can grab it. Same for the test suite: it wants the secret in config.js, but we can pull it in from wherever we like.

Thus, having CI or Quibble generate a new, random, strong secret for each test run seems like a good way of avoiding bad habits.

Indeed. So what's the best way to share the secret generated by quibble between two docker containers? Can you just make it show up in /run/secrets/mw-secret-key in both containers?

I'm rather blurry on the relationship between quibble and the container that runs the test framework. Or quibble's relationship with the container that runs MediaWiki, for that matter.

I continue to comment on this, while being ignorant about everything. If MW can read a secret from a file, I suggest this:

• CI or Quibble generates a random secret and stores it in /etc/supersecret in the container when it's built (new secret for every build)
• MW and the test software reads that file
• Tests are run as user x, group y, neither of which is root (I don't know what user/group is used now; they're probably fine)
• /etc/supersecret is owned by user root, group y, and is mode -rw-r----- (owner rw, group r, others nothing)

In T233092#5543922, @LarsWirzenius wrote:

I continue to comment on this, while being ignorant about everything. If MW can read a secret from a file, I suggest this:

I'll continue to reply, while being totally ignorant about quibble and the CI infrastructure.

• CI or Quibble generates a random secret and stores it in /etc/supersecret in the container when it's built (new secret for every build)
• MW and the test software reads that file

The test software runs in a *different* container. Does quibble build both? I don't know... Here we'll need someone who actually knows how the quibble controlled environment works :)

It's usually a bad idea to embed secrets into containers. It's better if the container can get the secret when it starts, but I'm not sure if our K8s cluster is set up for that. Or if these containers are run in K8s.

In T233092#5551316, @LarsWirzenius wrote:

It's usually a bad idea to embed secrets into containers. It's better if the container can get the secret when it starts, but I'm not sure if our K8s cluster is set up for that. Or if these containers are run in K8s.

So, who would be able to provide the necessary information?

Would it be possibly to use Jenkins' credential store? https://jenkins.io/doc/book/using/using-credentials/

From the quick code health chat we had with @zeljkofilipin @kostajh @daniel and me @hashar :

Feature request is for the api testing framework to trigger MediaWiki background jobs running and ensure they are complete. That is in order to later make assertion on the result of the execution of those jobs.

Special:RunJobs can be reached by the API testing framework and reliably run jobs (unlike hammering Main Page and checking jobs stats)

One could run maintenance/runJobs.php, which when running under Quibble can be located via the environment variable MW_INSTALL_PATH set to point to the root of mediawiki/core. It would only work for testing against a local instance of MediaWiki though.

Sharing $wgSecretKey would not be possible if one wants to run the api testing script against a remote wiki such as production or beta cluster. But another solution could be found to address that specific use case. Maybe by logging in and granting some privilege.

Anyway, the easiest way for now is what @daniel suggested, just have Quibble to set $wgSecretKey in LocalSettings.php. The setting is generated by the MediaWiki installer, although Quibble inserts custom settings, they are inserted at the top of the file and $wgSecretKey would thus be overriden by whatever value the installer has set it to.

Thus we need some logic to append config, which is what Quibble used to do ( 3add6e7fca5eccaae39f2fc9a79bde936f0195e6 ).

@awight has a similar need to turn on $wgEnableUploads to support some test that exercise uploading files ( T190829 , T199939 ). And https://gerrit.wikimedia.org/r/#/c/integration/quibble/+/540118/1 which inserts $wgEnableUploads = true; but would not work since that is inserted at the top.

Has the new version of quibble been deployed everywhere?
Should CI on the api-testing repo now be using the hardcoded secret?