unusual number of '-" pageviews in October
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Nuria
	Nov 17 2016, 9:25 PM

Description

The very high number of views of the "-" page is explained by this dash value being used as a special value for "no page title found" when extracting titles from urls.

This number in unusually hight in October 2016.

Details

	Subject	Repo	Branch	Lines +/-
	Adding self-identified bot to bot regex	analytics/refinery/source	master	+2 -1

Customize query in gerrit

Event Timeline

• Nuria created this task.Nov 17 2016, 9:25 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 17 2016, 9:25 PM

• Nuria claimed this task.Nov 17 2016, 9:27 PM

• Nuria edited projects, added Analytics-Kanban; removed Analytics.

Screen Shot 2016-11-18 at 12.07.29 PM.png (1×1 px, 1 MB)

Screenshoot from https://tools.wmflabs.org/pageviews/?project=en.wikipedia.org&platform=all-access&agent=user&start=2016-01-01&end=2016-11-05&pages=-

• Nuria moved this task from Next Up to In Progress on the Analytics-Kanban board.Nov 21 2016, 4:04 PM

The bulk of the "-" pageviews increase is due to traffic coming from this user agent: "Blackboard Safeassign" that looks to be a bot linked to this tool: http://www.safeassign.com/

"{"city":"Washington","country_code":"US"United States"} {"language_variant":"default","page_title":"-","project":"en.wikipedia"} /wiki ?curid=19792979 Blackboard Safeassign

Indeed located in washington DC: http://whois.domaintools.com/safeassign.com

Change 323249 had a related patch set uploaded (by Nuria):
Adding self-identified bot to bot regex

https://gerrit.wikimedia.org/r/323249

gerritbot added a project: Patch-For-Review.Nov 23 2016, 9:12 PM

• Nuria moved this task from In Progress to In Code Review on the Analytics-Kanban board.Nov 23 2016, 9:12 PM

• Nuria moved this task from In Code Review to Ready to Deploy on the Analytics-Kanban board.Nov 29 2016, 4:08 PM

Change 323249 merged by jenkins-bot:
Adding self-identified bot to bot regex

https://gerrit.wikimedia.org/r/323249

• Nuria moved this task from Ready to Deploy to Done on the Analytics-Kanban board.Nov 30 2016, 9:32 PM

Not that this is a good solution going forward but this particular bot has been addressed by our bot list: https://tools.wmflabs.org/pageviews/?project=en.wikipedia.org&platform=all-access&agent=user&start=2016-01-01&end=2016-12-03&pages=-

• Nuria closed this task as Resolved.Dec 4 2016, 8:47 PM