Page MenuHomePhabricator

Login on wikitech wiki fails after OpenStack upgrade removed v2 identity API
Closed, ResolvedPublic

Description

I can't login on Wikitech wiki anymore while the same (LDAP) credentials still work normal on Horizon.

https://wikitech.wikimedia.org/wiki/Main_Page

I am getting "Incorrect username or password entered. Please try again. Usernames are case-sensitive. See phab:T165795 for more details. "

@Paladox confirmed the same is the case for him while his credentials still work on Gerrit.

Event Timeline

Dzahn created this task.Tue, Oct 8, 8:07 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptTue, Oct 8, 8:07 PM
Paladox triaged this task as Unbreak Now! priority.Tue, Oct 8, 8:07 PM
Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptTue, Oct 8, 8:07 PM
Dzahn edited projects, added Operations, Cloud-Services; removed wikitech.wikimedia.org.
Dzahn added a subscriber: cloud-services-team.

Mentioned in SAL (#wikimedia-operations) [2019-10-08T20:24:30Z] <mutante> labweb1001 - edit /srv/mediawiki/wmf-config/wikitech.php to and change "false" to "true" on line 52 to enable LDAP debug logging for T234996

Dzahn added a comment.Tue, Oct 8, 8:28 PM

Enabled the debug log as suggested by Krenair.

Debug log shows a restCall to cloudcontrol1003 to get a token:

OpenStackNovaController::restCall fullurl: http://cloudcontrol1003.wikimedia.org

followed by:

OpenStackNovaController::authenticate return code: 401

401 = Unauthorized

Mentioned in SAL (#wikimedia-operations) [2019-10-08T20:38:51Z] <mutante> labweb1001 - disabled 2fa for myself on Wikitech using disableOATHAuthForUser.php --wiki=labswiki to debug T234996

Change 541644 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[mediawiki/extensions/OpenStackManager@master] OpenStack: switch to using the v3 keystone API for login

https://gerrit.wikimedia.org/r/541644

bd808 added a comment.Tue, Oct 8, 11:45 PM

I set a custom message at https://wikitech.wikimedia.org/wiki/MediaWiki:Loginprompt that will show up on the login screen.

bd808 renamed this task from login on wikitech wiki fails to Login on wikitech wiki fails after OpenStack upgrade removed v2 identity API.Tue, Oct 8, 11:57 PM
bd808 added a comment.Wed, Oct 9, 5:24 AM

Change 541644 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[mediawiki/extensions/OpenStackManager@master] OpenStack: switch to using the v3 keystone API for login
https://gerrit.wikimedia.org/r/541644

This patch is (at the moment of this comment) hot patched on labweb100{1,2}. With it in place I am able to login to Wikitech as the "BryanDavis" user. I also managed to replace all the v2 identity API calls.

bd808 added a comment.Wed, Oct 9, 5:25 AM

I set a custom message at https://wikitech.wikimedia.org/wiki/MediaWiki:Loginprompt that will show up on the login screen.

I deleted this custom message.

Dzahn added a comment.Thu, Oct 10, 2:51 PM

Could confirm yesterday i can login again with the hotfix. Thanks!

Change 542168 had a related patch set uploaded (by BryanDavis; owner: Andrew Bogott):
[mediawiki/extensions/OpenStackManager@wmf/1.35.0-wmf.1] OpenStack: switch to using the v3 keystone API for login

https://gerrit.wikimedia.org/r/542168

Change 541644 merged by jenkins-bot:
[mediawiki/extensions/OpenStackManager@master] OpenStack: switch to using the v3 keystone API for login

https://gerrit.wikimedia.org/r/541644

Change 542168 merged by jenkins-bot:
[mediawiki/extensions/OpenStackManager@wmf/1.35.0-wmf.1] OpenStack: switch to using the v3 keystone API for login

https://gerrit.wikimedia.org/r/542168

bd808 closed this task as Resolved.Thu, Oct 10, 7:25 PM
bd808 assigned this task to Andrew.
bd808 added subscribers: Andrew, dduvall.

@Andrew and I managed to tag team our way through updating OpenStackManager to use the v3 of OpenStack's identity API. @dduvall was nice enough to merge and deploy the backport for us.