Page MenuHomePhabricator

CAS update failed on user_touched for user ID
Closed, ResolvedPublic

Description

I saw this in the logs today

2015-04-12 08:00:15 translatewiki.net translatewiki_net-bw_: [ac0f8a96] /w/i.php?title=Special:Translate&group=ext-0-all&language=de&tux=0&limit=1000   MWException from line 3633 of /srv/mediawiki/tags/2015-04-10_10:04:17/includes/User.php: CAS update failed on user_touched for user ID '1509'.
#0 /srv/mediawiki/tags/2015-04-10_10:04:17/extensions/Translate/api/ApiTranslateUser.php(82): User->saveSettings()
#1 /srv/mediawiki/tags/2015-04-10_10:04:17/extensions/Translate/specials/SpecialTranslate.php(214): ApiTranslateUser::trackGroup(Object(AggregateMessageGroup), Object(User))
#2 /srv/mediawiki/tags/2015-04-10_10:04:17/includes/specialpage/SpecialPage.php(384): SpecialTranslate->execute(NULL)
#3 /srv/mediawiki/tags/2015-04-10_10:04:17/includes/specialpage/SpecialPageFactory.php(581): SpecialPage->run(NULL)
#4 /srv/mediawiki/tags/2015-04-10_10:04:17/includes/MediaWiki.php(267): SpecialPageFactory::executePath(Object(Title), Object(RequestContext))
#5 /srv/mediawiki/tags/2015-04-10_10:04:17/includes/MediaWiki.php(566): MediaWiki->performRequest()
#6 /srv/mediawiki/tags/2015-04-10_10:04:17/includes/MediaWiki.php(414): MediaWiki->main()
#7 /srv/mediawiki/tags/2015-04-10_10:04:17/index.php(46): MediaWiki->run()
  1. What does it mean? Preference was not updated?
  2. Is there a bug in my code?
  3. Is there a bug somewhere else?

Last 24 hours in Logstash:
https://logstash.wikimedia.org/goto/5b142cc357fafb79862828747f9b71f8

As of December 2017: 400+ hits per day.

Related Objects

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 233639 merged by jenkins-bot:
Use db-replicated objectcache for storing last seen time

https://gerrit.wikimedia.org/r/233639

aaron removed aaron as the assignee of this task.Sep 25 2015, 5:26 AM
aaron added a comment.Sep 25 2015, 5:59 AM

Kibana shows ~400 events/day over all wikis with the same users accounting for a good portion of errors. E.g grouping by message (which only varies by user):

5dfe33f20f9f286680b6750e08c58cdf (110) abfc11b8a055ee62a62e86484bd4a621 (23) ef435460868afdcc6e39817586c75ee8 (15) 9eed7fc251d2042d6f4269ee82e15601 (12) e8555fcc0b1fbac102fdf926a3e959ee (11) e4b05d5a29a01d828478f757969bf7b4 (9) 73403b212632c3c46be8dd7bf93fd51e (8) 5cc2f4daa3ac05efa4a2dff565451cd6 (6) 9a0cb924a677b87c15173729ad335120 (5) 259f4565301cd4ceebdfd91f7c8648c4 (5) Missing field (0) Other values (209)

Mostly double posts to ApiOptions or password reset.

Change 240968 had a related patch set uploaded (by Aaron Schulz):
Avoid CAS errors on double post to ApiOptions

https://gerrit.wikimedia.org/r/240968

Change 240968 merged by jenkins-bot:
Avoid CAS errors on double post to ApiOptions

https://gerrit.wikimedia.org/r/240968

Change 242718 had a related patch set uploaded (by Aaron Schulz):
Made User::loadFromId() skip cache with READ_LATEST

https://gerrit.wikimedia.org/r/242718

Change 242736 had a related patch set uploaded (by Ori.livneh):
Made User::loadFromId() skip cache with READ_LATEST

https://gerrit.wikimedia.org/r/242736

Change 242737 had a related patch set uploaded (by Ori.livneh):
Made User::loadFromId() skip cache with READ_LATEST

https://gerrit.wikimedia.org/r/242737

Change 242718 merged by jenkins-bot:
Made User::loadFromId() skip cache with READ_LATEST

https://gerrit.wikimedia.org/r/242718

Change 242736 merged by jenkins-bot:
Made User::loadFromId() skip cache with READ_LATEST

https://gerrit.wikimedia.org/r/242736

Change 242737 merged by jenkins-bot:
Made User::loadFromId() skip cache with READ_LATEST

https://gerrit.wikimedia.org/r/242737

Reedy added a subscriber: Reedy.Nov 29 2015, 8:29 PM
2015-11-29 20:24:04 mw1090 mediawikiwiki exception ERROR: [06e9ceb3] /wiki/Special:PasswordReset   MWException from line 3719 of /srv/mediawiki/php-1.27.0-wmf.7/includes/User.php: CAS update failed on user_touched for user ID '4315871' (read from slave); the version of the user to be saved is older than the current version. {"exception_id":"06e9ceb3"}

From 1.27.0-wmf.8:

CAS update failed on user_touched for user ID '***' (read from slave); the version of the user to be saved is older than the current version.
POST https://www.mediawiki.org/wiki/Special:PasswordReset

Change 258729 had a related patch set uploaded (by Aaron Schulz):
Avoid "CAS updated failed" errors on Special:Preferences double post

https://gerrit.wikimedia.org/r/258729

Change 258729 merged by Aaron Schulz:
Avoid "CAS updated failed" errors on Special:Preferences double post

https://gerrit.wikimedia.org/r/258729

Krinkle closed this task as Resolved.Dec 15 2015, 11:56 PM
Krinkle claimed this task.
Nikerabbit reopened this task as Open.Dec 16 2015, 1:43 PM

Only some causes are fixed, and the one in bug summary is not among them:

2015-12-16 13:18:17 translatewiki.net translatewiki_net-bw_: [410bb388] /w/i.php?title=Special:Translate&group=ext-0-all&language=de&tux=0&limit=500   MWException from line 3719 of /srv/mediawiki/tags/2015-12-16_07:54:21/includes/user/User.php: CAS update failed on user_touched for user ID '1509' (read from slave); the version of the user to be saved is older than the current version.
#0 /srv/mediawiki/tags/2015-12-16_07:54:21/extensions/Translate/api/ApiTranslateUser.php(83): User->saveSettings()
#1 /srv/mediawiki/tags/2015-12-16_07:54:21/includes/deferred/CallableUpdate.php(27): Closure$ApiTranslateUser::trackGroup()
#2 /srv/mediawiki/tags/2015-12-16_07:54:21/includes/deferred/DeferredUpdates.php(151): MWCallableUpdate->doUpdate()
#3 /srv/mediawiki/tags/2015-12-16_07:54:21/includes/deferred/DeferredUpdates.php(88): DeferredUpdates::execute(array, string)
#4 /srv/mediawiki/tags/2015-12-16_07:54:21/includes/MediaWiki.php(723): DeferredUpdates::doUpdates(string)
#5 /srv/mediawiki/tags/2015-12-16_07:54:21/includes/MediaWiki.php(570): MediaWiki->restInPeace(string)
#6 [internal function]: Closure$MediaWiki::doPostOutputShutdown()
#7 {main}
aaron removed aaron as the assignee of this task.Dec 16 2015, 11:51 PM

I think core should be pretty good now in wmf9/wfm10. I don't plan on improving a random extension that runs into this, though BetaFeatures might be worth a look.

Change 259967 had a related patch set uploaded (by Aaron Schulz):
Reduce CAS errors due to concurrent web requests by a user

https://gerrit.wikimedia.org/r/259967

Change 260533 had a related patch set uploaded (by Aaron Schulz):
Avoid CAS update errors on password reset

https://gerrit.wikimedia.org/r/260533

Change 260533 merged by jenkins-bot:
Avoid CAS update errors on password reset

https://gerrit.wikimedia.org/r/260533

Change 259967 merged by jenkins-bot:
Reduce CAS errors due to concurrent web requests by a user

https://gerrit.wikimedia.org/r/259967

Peachey88 updated the task description. (Show Details)Aug 8 2016, 12:10 AM
Andrew added a subscriber: Andrew.Nov 8 2016, 9:20 PM

I'm seeing quite a lot of these today -- I created 12 pages on wikitech at roughly the same time (each from different thread and/or host) and three of them failed.

Change 320482 had a related patch set uploaded (by Andrew Bogott):
wikistatus: work around occasional wikitech login failures

https://gerrit.wikimedia.org/r/320482

Change 320482 merged by Andrew Bogott:
wikistatus: work around occasional wikitech login failures

https://gerrit.wikimedia.org/r/320482

hashar added a subscriber: hashar.Nov 10 2016, 11:40 AM

Prod issue going on right now is T150373

Krinkle removed a subscriber: Krinkle.Nov 10 2016, 4:06 PM
hashar removed a subscriber: hashar.Dec 4 2017, 12:17 PM
demon added a subscriber: demon.Dec 4 2017, 8:21 PM

Does this need to be an exception, or would logging it suffice? That would allow grouping on the user IDs.

Noticed this happening in 1.31.0-wmf.16 just now.

Kghbln added a subscriber: Kghbln.Apr 13 2018, 3:15 PM

I am getting this for MLEB 2017-04 on MW 1.27.4 with PHP 7.0

When trying to save a translation I get:

Exception Caught: CAS update failed on user_touched for user ID '255' (read from slave); the version of the user to be saved is older than the current version.

From what I see here I believe patches were added to MW 1.27.0. So basically there is nothing I can do, even if I update to MW 1.27.4 and MLEB 2017.07?

To mitigate the issue I have to go the translation field again, make a trivial change and re-save.

It's not clear what is causing it to you. I have updated Translate long ago to avoid causing this kind of issues. But I don't run any 1.27 wikis so it is possible there is still something I might have missed.

Specifically, ApiTranslateUser was removed in 35772fb so based on that I would even close this bug because it addressed the original report.

Ok, as soon as 1.31 LTS is out I will also be able to upgrade to compatible versions, i.e. MLEB 2017.07 or later. If there is still trouble I will report back with a new issue. Thanks for your comment.

Nikerabbit closed this task as Resolved.Apr 27 2018, 3:37 PM
Nikerabbit claimed this task.

Thanks. I suggest that to file a new task in case you still observe issues.

Change 464072 had a related patch set uploaded (by Sophivorus; owner: Sophivorus):
[mediawiki/extensions/LastUserLogin@master] Bugfix "CAS update failed on user_touched for user ID"

https://gerrit.wikimedia.org/r/464072

Change 464072 merged by jenkins-bot:
[mediawiki/extensions/LastUserLogin@master] Bugfix "CAS update failed on user_touched for user ID"

https://gerrit.wikimedia.org/r/464072

mmodell changed the subtype of this task from "Task" to "Production Error".Wed, Aug 28, 11:12 PM