Page MenuHomePhabricator

notwikilambda Toolforge service down
Closed, ResolvedPublic

Description

https://notwikilambda.toolforge.org/ emits an HTTP 500. Nothing in the SAL. Did we merge something that broke it? Sorry!

Event Timeline

Hm, WSOAuth might need an update?

error.log
2022-01-24 17:06:53: (mod_fastcgi.c.421) FastCGI-stderr: PHP Fatal error:  Uncaught RuntimeException: The configuration setting 'wgPluggableAuth_ButtonLabelMessage' was already set by MediaWiki core or another extension, and cannot be set again by WSOAuth. in /data/project/notwikilambda/public_html/w/includes/registration/ExtensionProcessor.php:777

Could be related to https://gerrit.wikimedia.org/r/c/mediawiki/extensions/PluggableAuth/+/754987, merged yesterday. Apparently WSOAuth already defined the duplicate setting before, but it’s possible that MediaWiki only makes this an error when using manifest v2.

I’m a bit confused by what PluggableAuth and WSOAuth are doing here. It looks like this wgPluggableAuth_ButtonLabelMessage is a config that PluggableAuth expects the authentication provider to provide (hence “pluggable”)? But it doesn’t seem like extension.json is the right mechanism for it…

Mentioned in SAL (#wikimedia-cloud) [2022-01-24T17:24:13Z] <wm-bot> <lucaswerkmeister> kubectl delete deployment update # T299934

Mentioned in SAL (#wikimedia-cloud) [2022-01-24T17:25:06Z] <wm-bot> <lucaswerkmeister> git -C ~/public_html/w/extensions/PluggableAuth/ checkout 8b278afaca # T299934

Hm, that doesn’t seem to have helped at all.

Could be related to https://gerrit.wikimedia.org/r/c/mediawiki/extensions/PluggableAuth/+/754987, merged yesterday. Apparently WSOAuth already defined the duplicate setting before, but it’s possible that MediaWiki only makes this an error when using manifest v2.

The first failure in the log is fairly soon after that change was merged, though:

2022-01-23 19:31:03: (mod_fastcgi.c.421) FastCGI-stderr: PHP Fatal error:  Uncaught RuntimeException: The configuration setting 'wgPluggableAuth_ButtonLabelMessage' was already set by MediaWiki core or another extension, and cannot be set again by WSOAuth. in /data/project/notwikilambda/public_html/w/includes/registration/ExtensionProcessor.php:777

Mentioned in SAL (#wikimedia-cloud) [2022-01-24T17:34:21Z] <wm-bot> <lucaswerkmeister> git -C ~/public_html/w/extensions/PluggableAuth/ checkout cdf73daf11^ # T299934

Turns out it was https://gerrit.wikimedia.org/r/c/mediawiki/extensions/PluggableAuth/+/754104, a few commits earlier, but merged around the same time. Main page loads again. Leaving task open since this still needs to be resolved by the PluggableAuth / WSOAuth devs, I assume, and then at some point I’d like to turn on the automatic updates again.

Turns out it was https://gerrit.wikimedia.org/r/c/mediawiki/extensions/PluggableAuth/+/754104, a few commits earlier, but merged around the same time. Main page loads again. Leaving task open since this still needs to be resolved by the PluggableAuth / WSOAuth devs, I assume, and then at some point I’d like to turn on the automatic updates again.

Thank you!

Sorry to hear about the outage and thanks for troubleshooting!

Good to know that notwikilambda was/will again be autoupdating PluggableAuth. Due to the nature of the update to PluggableAuth, an update will required to all authentication plugins.

Agreed that extension.json is not the place to override the value.

@Xxmarijnw, tagging you to look at updating WSOAuth.

WSOAuth is currently not compatible with the latest version of PluggableAuth. I will update it as soon as possible and let you know through this task!

This is unfortunately taking longer than expected. I am currently very busy with other duties at Wikibase Solutions, and the update is more complex than I initially thought. I hope to have it finished in February, though!

Mentioned in SAL (#wikimedia-cloud) [2022-02-13T14:22:50Z] <wm-bot> <lucaswerkmeister> kubectl create -f ~/k8s/update/deployment.yaml # resume automatic updates except for extensions/PluggableAuth (T299934)

It seems that PluggableAuth was still automatically updated to master (6.0-dev), since logging in with WSOAuth is now failing.

It’s not on current master, it’s on rEPLGbc8cc95bf8b1: User LoggerFactory rather than wfDebugLog, which I thought was the last compatible commit (it’s the parent of rEPLGcdf73daf1198: Use main config rather than $GLOBALS, which seemed to be the commit that broke things). Should I switch to an earlier commit?

Mentioned in SAL (#wikimedia-cloud) [2022-02-25T09:04:48Z] <wm-bot> <lucaswerkmeister> git -C public_html/w/extensions/PluggableAuth/ checkout 520da2faa62349dcc3fd76c63df15f56ad15317d # T299934

Thanks, let’s see if that works better.

WSOAuth has now been updated to support PluggableAuth 6.0.

Please note that WSOAuth 6.0 is not backwards compatible with older versions as there have been some schema changes and the way to configure OAuth providers has changed. Please see the documentation (also take a look at Extension:WSOAuth#Upgrading from before 6.0).

Mentioned in SAL (#wikimedia-cloud) [2022-04-06T18:53:40Z] <wm-bot> <lucaswerkmeister> updated to PluggableAuth 6.0 (T299934)

Thanks, it seems to be working now.

Mentioned in SAL (#wikimedia-cloud) [2022-04-06T19:20:56Z] <wm-bot> <lucaswerkmeister> resumed automatic updates of PluggableAuth (T299934)