Change 904497 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/config@master] zuul: Add CentralAuth to gatedextensions

https://gerrit.wikimedia.org/r/904497

I've asked about this before, and my understanding is that the presence of CA completely breaks most of core's authentication tests.

Yeah, this was previously declined sadly for that reason. :-( In the magical post-GitLab world I suppose we could build a new CI runner that runs multiple configuration suites in parallel, with and without CentralAuth?

In T333541#8741475, @taavi wrote:

I've asked about this before, and my understanding is that the presence of CA completely breaks most of core's authentication tests.

CentralAuth replaces mediawiki/core authentication system, that breaks other extensions (and mediawiki authentication related tests as well). Adding CentralAuth has been proposed previously via T321864 and declined. There are also poor interactions between extensions which prevent us from adding some cause they will break test for other extensions.

That being said, we could imagine a system in which mediawiki/core auth related tests will set a known working state to prevent CentralAuth code from interacting. Then I have no idea how CentralAuth overrides MediaWiki authentication system, if it is solely via hooks and config system, I guess the mediawiki/core tests can set them to known good values discarding the CentralAuth config.

We can add an experimental job at least if one wants to investigate.

Change 904497 abandoned by Hashar:

[integration/config@master] zuul: Add CentralAuth to gatedextensions

Reason:

https://gerrit.wikimedia.org/r/904497

I suspect T358985: Admin account created by the installer isn't made global by CentralAuth is one thing we'd have to figure out to avoid causing issues in other extensions' tests.

Can I suggest either re-purposing this task, or creating a different one, for "Provide a CI job that runs against all MediaWiki patches and stops them from merging if it breaks CentralAuth's tests"? As CA is such a critical part of our infrastructure, maybe we should just create a bespoke job for that?

Isn't that the same thing as adding it to the gated extensions? Or do you mean MediaWiki core patches only?

Historically I think the concern was slowing CI down. Now that tests are parallelized, that's probably not that much of an issue.

In T333541#11243876, @Tgr wrote:

Isn't that the same thing as adding it to the gated extensions? Or do you mean MediaWiki core patches only?

No, I mean a new CI job, not adding it to the list in gated extensions.

Historically I think the concern was slowing CI down. Now that tests are parallelized, that's probably not that much of an issue.

No, that's the general concern about the gate; for CA, the problem is that CA disables regular MW login and so (a) stops us from testing critical functionality in MW, like log-in to officewiki, and (b) can break some extensions' tests (IIRC, browser tests are affected).

Right, the PHPUnit code overrides CentralAuth and restores the core authentication provider, but that does not help browser tests. (Plus, probably there are still minor ways where CentralAuth might conflict with other things, via hooks etc.)

I still think we should work out how to run CentralAuth in a more compatible way, as there have been examples of patches to other extensions breaking CentralAuth (and occasionally causing problems in production that took an embarrassingly long time to notice and fix), so running CA tests on core patches isn't always sufficient. But that's a more complicated problem.

Filed as T406582: Block the merging of patches to MediaWiki core that would break CentralAuth tests.