Page MenuHomePhabricator
Create Task
Maniphest T140144

Echo triggering CentralAuth "Can only obtain a centralauthtoken when using CentralAuth sessions" error
Closed, ResolvedPublic
Actions

Description

We're still getting:

2016-07-12 18:38:21 [V4U5HApAMDgAACyfYbEAAAAX] mw1221 commonswiki 1.28.0-wmf.8 exception ERROR: [V4U5HApAMDgAACyfYbEAAAAX] /w/api.php?action=query&format=json&origin=https%3A%2F%2Fes%2Ewikipedia%2Eorg&centralauthtoken=7b1aae48ebbcd1df01a7c0202a45c59e2cfeec6&meta=siteinfo&siprop=uploaddialog&formatversion=2   UsageException from line 1487 of /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiBase.php: Can only obtain a centralauthtoken when using CentralAuth sessions {"exception_id":"V4U5HApAMDgAACyfYbEAAAAX"} 
[Exception UsageException] (/srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiBase.php:1487) Can only obtain a centralauthtoken when using CentralAuth sessions
  #0 /srv/mediawiki/php-1.28.0-wmf.8/extensions/CentralAuth/includes/api/ApiCentralAuthToken.php(51): ApiBase->dieUsage(string, string)
  #1 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(1373): ApiCentralAuthToken->execute()
  #2 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(438): ApiMain->executeAction()
  #3 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(49): ApiMain->execute()
  #4 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(88): EchoForeignWikiRequest->getCentralAuthToken(User)
  #5 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(68): EchoForeignWikiRequest->getQueryParams(string)
  #6 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(29): EchoForeignWikiRequest->getRequestParams()
  #7 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(709): EchoForeignWikiRequest->execute()
  #8 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(740): MWEchoNotifUser->getForeignData()
  #9 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(487): MWEchoNotifUser->getForeignCount(string)
  #10 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/Hooks.php(1150): MWEchoNotifUser->resetNotificationCount()
  #11 /srv/mediawiki/php-1.28.0-wmf.8/includes/Hooks.php(195): EchoHooks::onUserSaveSettings(User)
  #12 /srv/mediawiki/php-1.28.0-wmf.8/includes/user/User.php(3952): Hooks::run(string, array)
  #13 /srv/mediawiki/php-1.28.0-wmf.8/includes/auth/AuthManager.php(1678): User->saveSettings()
  #14 /srv/mediawiki/php-1.28.0-wmf.8/includes/Setup.php(861): MediaWiki\Auth\AuthManager->autoCreateUser(User, string, boolean)
  #15 /srv/mediawiki/php-1.28.0-wmf.8/includes/WebStart.php(137): include(string)
  #16 /srv/mediawiki/php-1.28.0-wmf.8/api.php(38): include(string)
  #17 /srv/mediawiki/w/api.php(3): include(string)
  #18 {main}

I'm putting up a troubleshooting patch.

It does happen when there is a central ID (included more just to show there are multiple entry points).

2016-07-12 19:30:15 [V4VFRwpAEDIAALPruqIAAAAD] mw1285 commonswiki 1.28.0-wmf.8 Echo DEBUG: Exception when fetching CentralAuth token: wiki: commonswiki, userName: Laurenthomp, userId: 6018965, centralId: 47181900, exception: badsession: Can only obtain a centralauthtoken when using CentralAuth sessions {"wiki":"commonswiki","userName":"Laurenthomp","userId":6018965,"centralId":47181900} 
[Exception UsageException] (/srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiBase.php:1487) Can only obtain a centralauthtoken when using CentralAuth sessions
  #0 /srv/mediawiki/php-1.28.0-wmf.8/extensions/CentralAuth/includes/api/ApiCentralAuthToken.php(51): ApiBase->dieUsage(string, string)
  #1 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(1373): ApiCentralAuthToken->execute()
  #2 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(438): ApiMain->executeAction()
  #3 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(57): ApiMain->execute()
  #4 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(112): EchoForeignWikiRequest->getCentralAuthToken(User)
  #5 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(92): EchoForeignWikiRequest->getQueryParams(string)
  #6 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(29): EchoForeignWikiRequest->getRequestParams()
  #7 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(709): EchoForeignWikiRequest->execute()
  #8 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(740): MWEchoNotifUser->getForeignData()
  #9 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(487): MWEchoNotifUser->getForeignCount(string)
  #10 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/Hooks.php(1150): MWEchoNotifUser->resetNotificationCount()
  #11 /srv/mediawiki/php-1.28.0-wmf.8/includes/Hooks.php(195): EchoHooks::onUserSaveSettings(User)
  #12 /srv/mediawiki/php-1.28.0-wmf.8/includes/user/User.php(3952): Hooks::run(string, array)
  #13 /srv/mediawiki/php-1.28.0-wmf.8/includes/auth/AuthManager.php(1678): User->saveSettings()
  #14 /srv/mediawiki/php-1.28.0-wmf.8/includes/Setup.php(861): MediaWiki\Auth\AuthManager->autoCreateUser(User, string, boolean)
  #15 /srv/mediawiki/php-1.28.0-wmf.8/includes/WebStart.php(137): include(string)
  #16 /srv/mediawiki/php-1.28.0-wmf.8/api.php(38): include(string)
  #17 /srv/mediawiki/w/api.php(3): include(string)
  #18 {main}
2016-07-12 19:30:16 [V4VFRwpAEDIAALPruqIAAAAD] mw1285 commonswiki 1.28.0-wmf.8 Echo DEBUG: Exception when fetching CentralAuth token: wiki: commonswiki, userName: Laurenthomp, userId: 6018965, centralId: 47181900, exception: badsession: Can only obtain a centralauthtoken when using CentralAuth sessions {"wiki":"commonswiki","userName":"Laurenthomp","userId":6018965,"centralId":47181900} 
[Exception UsageException] (/srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiBase.php:1487) Can only obtain a centralauthtoken when using CentralAuth sessions
  #0 /srv/mediawiki/php-1.28.0-wmf.8/extensions/CentralAuth/includes/api/ApiCentralAuthToken.php(51): ApiBase->dieUsage(string, string)
  #1 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(1373): ApiCentralAuthToken->execute()
  #2 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(438): ApiMain->executeAction()
  #3 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(57): ApiMain->execute()
  #4 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(112): EchoForeignWikiRequest->getCentralAuthToken(User)
  #5 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(92): EchoForeignWikiRequest->getQueryParams(string)
  #6 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(29): EchoForeignWikiRequest->getRequestParams()
  #7 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(709): EchoForeignWikiRequest->execute()
  #8 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(740): MWEchoNotifUser->getForeignData()
  #9 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(487): MWEchoNotifUser->getForeignCount(string)
  #10 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/model/Notification.php(161): MWEchoNotifUser->resetNotificationCount(integer)
  #11 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/mapper/NotificationMapper.php(61): Closure$EchoNotification::insert#2()
  #12 /srv/mediawiki/php-1.28.0-wmf.8/includes/db/Database.php(2482): Closure$EchoNotificationMapper::insert()
  #13 /srv/mediawiki/php-1.28.0-wmf.8/includes/db/Database.php(2454): DatabaseBase->runOnTransactionIdleCallbacks()
  #14 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/mapper/NotificationMapper.php(64): DatabaseBase->onTransactionIdle(Closure$EchoNotificationMapper::insert;1964115705)
  #15 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/model/Notification.php(165): EchoNotificationMapper->insert(EchoNotification)
  #16 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/model/Notification.php(94): EchoNotification->insert()
  #17 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/Notifier.php(21): EchoNotification::create(array)
  #18 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/controller/NotificationController.php(277): EchoNotifier::notifyWithNotification(User, EchoEvent)
  #19 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/controller/NotificationController.php(104): EchoNotificationController::doNotification(EchoEvent, User, string)
  #20 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/model/Event.php(155): EchoNotificationController::notify(EchoEvent, boolean)
  #21 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/DiscussionParser.php(72): EchoEvent::create(array)
  #22 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/Hooks.php(474): EchoDiscussionParser::generateEventsForRevision(Revision)
  #23 /srv/mediawiki/php-1.28.0-wmf.8/includes/deferred/CallableUpdate.php(22): Closure$EchoHooks::onArticleSaved()
  #24 /srv/mediawiki/php-1.28.0-wmf.8/includes/deferred/DeferredUpdates.php(159): MWCallableUpdate->doUpdate()
  #25 /srv/mediawiki/php-1.28.0-wmf.8/includes/deferred/DeferredUpdates.php(88): DeferredUpdates::execute(array, string)
  #26 /srv/mediawiki/php-1.28.0-wmf.8/includes/MediaWiki.php(776): DeferredUpdates::doUpdates(string)
  #27 /srv/mediawiki/php-1.28.0-wmf.8/includes/MediaWiki.php(618): MediaWiki->restInPeace(string)
  #28 [internal function]: Closure$MediaWiki::doPostOutputShutdown()
  #29 {main}
2016-07-12 19:31:16 [V4VFhApAMEAAACPlaaYAAABQ] mw1229 commonswiki 1.28.0-wmf.8 Echo DEBUG: Exception when fetching CentralAuth token: wiki: commonswiki, userName: Antonio wimoveis, userId: 6
018344, centralId: 47129486, exception: badsession: Can only obtain a centralauthtoken when using CentralAuth sessions {"wiki":"commonswiki","userName":"Antonio wimoveis","userId":6018344,"c
entralId":47129486} 
[Exception UsageException] (/srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiBase.php:1487) Can only obtain a centralauthtoken when using CentralAuth sessions
  #0 /srv/mediawiki/php-1.28.0-wmf.8/extensions/CentralAuth/includes/api/ApiCentralAuthToken.php(51): ApiBase->dieUsage(string, string)
  #1 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(1373): ApiCentralAuthToken->execute()
  #2 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(438): ApiMain->executeAction()
  #3 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(57): ApiMain->execute()
  #4 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(112): EchoForeignWikiRequest->getCentralAuthToken(User)
  #5 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(92): EchoForeignWikiRequest->getQueryParams(string)
  #6 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(29): EchoForeignWikiRequest->getRequestParams()
  #7 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(709): EchoForeignWikiRequest->execute()
  #8 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(770): MWEchoNotifUser->getForeignData()
  #9 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(492): MWEchoNotifUser->getForeignTimestamp(string)
  #10 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(365): MWEchoNotifUser->resetNotificationCount(integer)
  #11 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/api/ApiEchoMarkRead.php(22): MWEchoNotifUser->markRead(array)
  #12 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(1373): ApiEchoMarkRead->execute()
  #13 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(469): ApiMain->executeAction()
  #14 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(440): ApiMain->executeActionWithErrorHandling()
  #15 /srv/mediawiki/php-1.28.0-wmf.8/api.php(83): ApiMain->execute()
  #16 /srv/mediawiki/w/api.php(3): include(string)
  #17 {main}

Details

SubjectRepoBranchLines +/-
CentralAuth: Bail if not fully initializedmediawiki/extensions/Echowmf/1.28.0-wmf.9+8 -3
CentralAuth: Bail if not fully initializedmediawiki/extensions/Echowmf/1.28.0-wmf.8+8 -3
CentralAuth: Bail if not fully initializedmediawiki/extensions/Echomaster+8 -3
Troubleshoot why Echo is still triggering CA failuresmediawiki/extensions/Echowmf/1.28.0-wmf.9+31 -7
Troubleshoot why Echo is still triggering CA failuresmediawiki/extensions/Echowmf/1.28.0-wmf.8+31 -7
Troubleshoot why Echo is still triggering CA failuresmediawiki/extensions/Echomaster+31 -7
Customize query in gerrit

Related Objects
Search...

Event Timeline

• Mattflaschen-WMF created this task.Jul 12 2016, 6:50 PM
Restricted Application removed a project: Patch-For-Review. · View Herald TranscriptJul 12 2016, 6:50 PM
Legoktm edited projects, added Notifications; removed MW-1.28-release-notes, MW-1.28-release (WMF-deploy-2016-06-28_(1.28.0-wmf.8)), MW-1.28-release (WMF-deploy-2016-07-05_(1.28.0-wmf.9)), MW-1.28-release (WMF-deploy-2016-07-12_(1.28.0-wmf.10)), MW-1.27-release (WMF-deploy-2016-01-12_(1.27.0-wmf.10)), WMF-General-or-Unknown, MediaWiki-User-login-and-signup.Jul 12 2016, 6:52 PM
gerritbot added a comment.Jul 12 2016, 6:54 PM

Change 298551 had a related patch set uploaded (by Mattflaschen):
Troubleshoot why Echo is still triggering CA failures

https://gerrit.wikimedia.org/r/298551

gerritbot added a project: Patch-For-Review.Jul 12 2016, 6:54 PM
gerritbot added a comment.Jul 12 2016, 7:15 PM

Change 298551 merged by jenkins-bot:
Troubleshoot why Echo is still triggering CA failures

https://gerrit.wikimedia.org/r/298551

gerritbot added a comment.Jul 12 2016, 7:17 PM

Change 298553 had a related patch set uploaded (by Mattflaschen):
Troubleshoot why Echo is still triggering CA failures

https://gerrit.wikimedia.org/r/298553

gerritbot added a comment.Jul 12 2016, 7:17 PM

Change 298554 had a related patch set uploaded (by Mattflaschen):
Troubleshoot why Echo is still triggering CA failures

https://gerrit.wikimedia.org/r/298554

gerritbot added a comment.Jul 12 2016, 7:25 PM

Change 298553 merged by jenkins-bot:
Troubleshoot why Echo is still triggering CA failures

https://gerrit.wikimedia.org/r/298553

gerritbot added a comment.Jul 12 2016, 7:25 PM

Change 298554 merged by jenkins-bot:
Troubleshoot why Echo is still triggering CA failures

https://gerrit.wikimedia.org/r/298554

Stashbot mentioned this in T119736: Could not find local user data for {Username}@{wiki}.Jul 12 2016, 7:30 PM

Mentioned in SAL [2016-07-12T19:30:11Z] <mattflaschen@tin> Synchronized php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php: T119736: T140144: Troubleshoot why Echo is still triggering CA failures (duration: 00m 39s)

• Mattflaschen-WMF added a project: Collab-Team-Q1-July-Sep-2016.Jul 12 2016, 7:31 PM
• Mattflaschen-WMF updated the task description. (Show Details)Jul 12 2016, 7:45 PM

So it does happen even when CentralAuth consistently gives us a central ID.

• Mattflaschen-WMF updated the task description. (Show Details)Jul 12 2016, 7:49 PM
• Mattflaschen-WMF added a comment.Jul 12 2016, 7:52 PM

The failures are not just during autoCreate, but that is the most serious since it causes corruption (missing user row in wiki DB), and still happening.

• Mattflaschen-WMF added a comment.EditedJul 12 2016, 7:53 PM

There is also one about "invalid key type". Not sure if that one happens in auto-create yet. (it is an autocreate, but a different path).

2016-07-12 19:49:17 [V4VJugpAIDEAAGRzeuoAAABN] mw1306 mediawikiwiki 1.28.0-wmf.8 Echo DEBUG: Exception when fetching CentralAuth token: wiki: mediawikiwiki, userName: Jtnayem147, userId: 684
7810, centralId: 47182055, exception: exception 'InvalidArgumentException' with message 'Invalid key type: NULL' in /srv/mediawiki/php-1.28.0-wmf.8/includes/utils/MWCryptHash.php:110
Stack trace:
#0 /srv/mediawiki/php-1.28.0-wmf.8/extensions/CentralAuth/includes/CentralAuthUser.php(607): MWCryptHash::hmac()
#1 /srv/mediawiki/php-1.28.0-wmf.8/extensions/CentralAuth/includes/api/ApiCentralAuthToken.php(62): CentralAuthUser->getAuthToken()
#2 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(1373): ApiCentralAuthToken->execute()
#3 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(438): ApiMain->executeAction()
#4 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(57): ApiMain->execute()
#5 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(112): EchoForeignWikiRequest->getCentralAuthToken()
#6 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(92): EchoForeignWikiRequest->getQueryParams()
#7 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(29): EchoForeignWikiRequest->getRequestParams()
#8 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(709): EchoForeignWikiRequest->execute()
#9 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(770): MWEchoNotifUser->getForeignData()
#10 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(496): MWEchoNotifUser->getForeignTimestamp()
#11 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/Hooks.php(1150): MWEchoNotifUser->resetNotificationCount()
#12 /srv/mediawiki/php-1.28.0-wmf.8/includes/Hooks.php(195): EchoHooks::onUserSaveSettings()
#13 /srv/mediawiki/php-1.28.0-wmf.8/includes/user/User.php(3952): Hooks::run()
#14 /srv/mediawiki/php-1.28.0-wmf.8/includes/auth/AuthManager.php(1678): User->saveSettings()
#15 /srv/mediawiki/php-1.28.0-wmf.8/extensions/CentralAuth/includes/CentralAuthUtils.php(141): MediaWiki\Auth\AuthManager->autoCreateUser()
#16 /srv/mediawiki/php-1.28.0-wmf.8/extensions/CentralAuth/includes/CreateLocalAccountJob.php(56): CentralAuthUtils::autoCreateUser()
#17 /srv/mediawiki/php-1.28.0-wmf.8/includes/jobqueue/JobRunner.php(265): CentralAuthCreateLocalAccountJob->run()
#18 /srv/mediawiki/php-1.28.0-wmf.8/includes/jobqueue/JobRunner.php(179): JobRunner->executeJob()
#19 /srv/mediawiki/rpc/RunJobs.php(47): JobRunner->run()
#20 {main} {"wiki":"mediawikiwiki","userName":"Jtnayem147","userId":6847810,"centralId":47182055} 
[Exception InvalidArgumentException] (/srv/mediawiki/php-1.28.0-wmf.8/includes/utils/MWCryptHash.php:110) Invalid key type: NULL
  #0 /srv/mediawiki/php-1.28.0-wmf.8/extensions/CentralAuth/includes/CentralAuthUser.php(607): MWCryptHash::hmac(string, NULL, boolean)
  #1 /srv/mediawiki/php-1.28.0-wmf.8/extensions/CentralAuth/includes/api/ApiCentralAuthToken.php(62): CentralAuthUser->getAuthToken()
  #2 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(1373): ApiCentralAuthToken->execute()
  #3 /srv/mediawiki/php-1.28.0-wmf.8/includes/api/ApiMain.php(438): ApiMain->executeAction()
  #4 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(57): ApiMain->execute()
  #5 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(112): EchoForeignWikiRequest->getCentralAuthToken(User)
  #6 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(92): EchoForeignWikiRequest->getQueryParams(string)
  #7 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php(29): EchoForeignWikiRequest->getRequestParams()
  #8 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(709): EchoForeignWikiRequest->execute()
  #9 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(770): MWEchoNotifUser->getForeignData()
  #10 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/includes/NotifUser.php(496): MWEchoNotifUser->getForeignTimestamp(string)
  #11 /srv/mediawiki/php-1.28.0-wmf.8/extensions/Echo/Hooks.php(1150): MWEchoNotifUser->resetNotificationCount()
  #12 /srv/mediawiki/php-1.28.0-wmf.8/includes/Hooks.php(195): EchoHooks::onUserSaveSettings(User)
  #13 /srv/mediawiki/php-1.28.0-wmf.8/includes/user/User.php(3952): Hooks::run(string, array)
  #14 /srv/mediawiki/php-1.28.0-wmf.8/includes/auth/AuthManager.php(1678): User->saveSettings()
  #15 /srv/mediawiki/php-1.28.0-wmf.8/extensions/CentralAuth/includes/CentralAuthUtils.php(141): MediaWiki\Auth\AuthManager->autoCreateUser(User, string, boolean)
  #16 /srv/mediawiki/php-1.28.0-wmf.8/extensions/CentralAuth/includes/CreateLocalAccountJob.php(56): CentralAuthUtils::autoCreateUser(User)
  #17 /srv/mediawiki/php-1.28.0-wmf.8/includes/jobqueue/JobRunner.php(265): CentralAuthCreateLocalAccountJob->run()
  #18 /srv/mediawiki/php-1.28.0-wmf.8/includes/jobqueue/JobRunner.php(179): JobRunner->executeJob(CentralAuthCreateLocalAccountJob, BufferingStatsdDataFactory, integer)
  #19 /srv/mediawiki/rpc/RunJobs.php(47): JobRunner->run(array)
  #20 {main}
• Mattflaschen-WMF updated the task description. (Show Details)Jul 12 2016, 7:58 PM
gerritbot added a comment.Jul 12 2016, 8:05 PM

Change 298569 had a related patch set uploaded (by Mattflaschen):
CentralAuth: Bail if not fully initialized

https://gerrit.wikimedia.org/r/298569

gerritbot added a comment.Jul 12 2016, 8:57 PM

Change 298569 merged by jenkins-bot:
CentralAuth: Bail if not fully initialized

https://gerrit.wikimedia.org/r/298569

ReleaseTaggerBot added a project: MW-1.28-release (WMF-deploy-2016-07-12_(1.28.0-wmf.10)).Jul 12 2016, 9:00 PM
gerritbot added a comment.Jul 12 2016, 9:03 PM

Change 298628 had a related patch set uploaded (by Mattflaschen):
CentralAuth: Bail if not fully initialized

https://gerrit.wikimedia.org/r/298628

Michgrig unsubscribed.Jul 12 2016, 9:04 PM
gerritbot added a comment.Jul 12 2016, 9:12 PM

Change 298634 had a related patch set uploaded (by Mattflaschen):
CentralAuth: Bail if not fully initialized

https://gerrit.wikimedia.org/r/298634

gerritbot added a comment.Jul 12 2016, 9:19 PM

Change 298628 merged by jenkins-bot:
CentralAuth: Bail if not fully initialized

https://gerrit.wikimedia.org/r/298628

gerritbot added a comment.Jul 12 2016, 9:21 PM

Change 298634 merged by jenkins-bot:
CentralAuth: Bail if not fully initialized

https://gerrit.wikimedia.org/r/298634

Stashbot added a comment.Jul 12 2016, 9:24 PM

Mentioned in SAL [2016-07-12T21:24:21Z] <mattflaschen@tin> Synchronized php-1.28.0-wmf.8/extensions/Echo/includes/ForeignWikiRequest.php: T140144: Echo/CentralAuth: Bail if not fully initialized (duration: 00m 49s)

ReleaseTaggerBot added projects: MW-1.28-release (WMF-deploy-2016-07-05_(1.28.0-wmf.9)), MW-1.28-release (WMF-deploy-2016-06-28_(1.28.0-wmf.8)).Jul 12 2016, 10:00 PM
• Mattflaschen-WMF added a comment.Jul 12 2016, 10:43 PM

There haven't been any more badsession errors since this was deployed, so I'm declaring it fixed.

T140156: CentralAuth 'Invalid key type: NULL' remains an issue, but I think that is on the CentralAuth side.

Also, it now catches the exception. So if it still fails to get a token, it should lead to "Failed to fetch API response", but not an uncaught exception or DB inconsistency.

• Mattflaschen-WMF moved this task from Untriaged to QA Review on the Collab-Team-Q1-July-Sep-2016 board.Jul 12 2016, 11:17 PM
jayvdb subscribed.Jul 13 2016, 12:23 AM
• Mattflaschen-WMF added a comment.Jul 13 2016, 6:55 PM

Overview of logs for the Echo part: P3422

Keep in mind the dates of the files are the date of log rotation, not necessarily date of occurrence.

• Mattflaschen-WMF added a comment.EditedJul 13 2016, 8:01 PM

@Legoktm is running checkLocalUser.php now, which should fix any users from the last stages of when the Echo bug was present. Current run started 2016-07-13 18:22 UTC.

greg added a comment.Jul 14 2016, 6:24 AM
In T140144#2455822, @Mattflaschen-WMF wrote:

There haven't been any more badsession errors since this was deployed, so I'm declaring it fixed.

Shall we close this then? Or are you waiting for (second) the checkLocaluser.php run to finish?

T140156: CentralAuth 'Invalid key type: NULL' remains an issue, but I think that is on the CentralAuth side.

Do you/does anyone have an idea of how prevalent it is?

greg mentioned this in T140156: CentralAuth 'Invalid key type: NULL'.Jul 14 2016, 6:26 AM
• Mattflaschen-WMF added a comment.Jul 14 2016, 6:56 PM
In T140144#2460924, @greg wrote:
In T140144#2455822, @Mattflaschen-WMF wrote:

There haven't been any more badsession errors since this was deployed, so I'm declaring it fixed.

Shall we close this then? Or are you waiting for (second) the checkLocaluser.php run to finish?

This is just waiting on QA now.

Etonkovidova subscribed.Jul 14 2016, 11:23 PM

Tested betalabs with the scenarios that supposedly trigger the error:
(1)

  • a newly created user goes to another wiki (not the wiki where the account was created)
  • an old user goes to a never before visited wikis
  • testing when different (tabs, windows, browsers) are open for the same account and some updated in message count/status happen only in one tab.

(2) Checked betalabs logs with @Mattflaschen-WMF grep
zgrep -c 'Can only obtain a centralauthtoken when using CentralAuth sessions' /a/mw-log/archive/exception.log-20160*
The warning was recorded between 2016-05-01 and 2016-07-11 (not later)

Etonkovidova moved this task from QA Review to Product Review on the Collab-Team-Q1-July-Sep-2016 board.Jul 14 2016, 11:24 PM
greg lowered the priority of this task from Unbreak Now! to High.Jul 15 2016, 3:26 PM
In T140144#2463455, @Mattflaschen-WMF wrote:
In T140144#2460924, @greg wrote:
In T140144#2455822, @Mattflaschen-WMF wrote:

There haven't been any more badsession errors since this was deployed, so I'm declaring it fixed.

Shall we close this then? Or are you waiting for (second) the checkLocaluser.php run to finish?

This is just waiting on QA now.

Per ^ reducing priority (just because it's in the blocker chain for wmf.10).

• jmatazzoni closed this task as Resolved.Jul 20 2016, 9:00 PM
Liuxinyu970226 unsubscribed.Jul 21 2016, 12:37 AM
MarcoAurelio moved this task from Backlog to Done on the MediaWiki-extensions-CentralAuth board.Dec 15 2016, 4:09 PM
Aklapper removed subscribers: Anomie, • Mattflaschen-WMF.Oct 16 2020, 5:40 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL