Maniphest T109797

Investigate bad_token statsd spikes
Closed, DeclinedPublic
Actions

Assigned To

None

Authored By

	aaron
	Aug 21 2015, 12:34 AM

Tags

Referenced Files

	F4953269: Screen Shot 2016-12-02 at 22.45.12.png
	Dec 3 2016, 6:46 AM

	F4953230: Screen Shot 2016-12-02 at 22.40.50.png
	Dec 3 2016, 6:43 AM

	F4953212: Screen Shot 2016-12-02 at 22.39.23.png
	Dec 3 2016, 6:43 AM

Subscribers

Description

http://graphite.wikimedia.org/render/?width=887&height=556&_salt=1440105991.976&from=-4weeks&target=divideSeries(MediaWiki.edit.failures.bad_token.count,MediaWiki.site.edits.count)&lineMode=connected&drawNullAsZero=false

This metric has been spikey, with waves of activity for it's lifetime. It's not immediately obvious why we'd get bad token errors in these patterns.

Related Objects

Mentioned Here: T141557: edit.failures.session_loss regressed on Wikimedia cluster

Event Timeline

aaron created this task.Aug 21 2015, 12:34 AM

aaron raised the priority of this task from to Needs Triage.

aaron updated the task description. (Show Details)

aaron added projects: Sustainability, MediaWiki-Page-editing, VisualEditor.

aaron subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 21 2015, 12:34 AM

Jdforrester-WMF triaged this task as Medium priority.Aug 21 2015, 8:08 PM

Jdforrester-WMF added projects: Contributors-Team, Performance-Team.

Jdforrester-WMF set Security to None.

Jdforrester-WMF moved this task from To Triage to Freezer on the VisualEditor board.Aug 25 2015, 7:04 PM

Krinkle moved this task from Inbox, needs triage to To-do: Goals prioritized current Quarter on the Performance-Team board.Sep 10 2015, 2:30 AM

May be related: T141557: edit.failures.session_loss regressed on Wikimedia cluster

Krinkle moved this task from To-do: Goals prioritized current Quarter to Radar on the Performance-Team board.Jul 28 2016, 9:01 PM

<http://graphite.wikimedia.org/render/?width=887&height=556&from=-4weeks&target=divideSeries(MediaWiki.edit.failures.bad_token.count,MediaWiki.site.edits.count)&lineMode=connected&drawNullAsZero=false

The .count properties represent the number of packets received by statsd, not their value. So in a minute that saw three +0 values, three +1, three +10, all get recorded as a 3 in count.

Data from August 2015 appears to no longer be in Graphte for these properties for some reason (only goes back to December 2015).

More recent query:

Screen Shot 2016-12-02 at 22.39.23.png (496×1 px, 81 KB)

And at https://grafana.wikimedia.org/dashboard/db/edit-count:

Screen Shot 2016-12-02 at 22.45.12.png (898×2 px, 323 KB)

Closing for now as bad_token is fairly rare.