Page MenuHomePhabricator

Add regexps that match the bots that follow the User-Agent policy {hawk}
Closed, ResolvedPublic3 Story Points

Description

Currently, the PageviewDefinition.java does not have regular expressions that match the bots that follow the conventions specified in https://meta.wikimedia.org/wiki/User-Agent_policy. We should implement this, and tag them as "bot" (or "spider" :-), it's a low hanging fruit I think.

Event Timeline

mforns created this task.Feb 3 2016, 11:04 PM
mforns updated the task description. (Show Details)
mforns raised the priority of this task from to Needs Triage.
mforns added a project: Analytics.
mforns added a subscriber: mforns.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptFeb 3 2016, 11:04 PM
jayvdb added a subscriber: jayvdb.Feb 3 2016, 11:14 PM
elukey added a subscriber: elukey.Feb 4 2016, 8:51 AM
Milimetric triaged this task as Normal priority.
Milimetric set Security to None.
Milimetric added a subscriber: Milimetric.

research whether the new expressions that we're adding have a match on the cluster

Nuria edited projects, added Analytics-Kanban; removed Analytics.Feb 25 2016, 5:34 PM
Nuria added a subscriber: Nuria.

Moving this task to kanban so it is up for grabs given that it is pretty small

Milimetric set the point value for this task to 3.Mar 3 2016, 5:38 PM
madhuvishy claimed this task.

Change 275760 had a related patch set uploaded (by Madhuvishy):
Implement the Wikimedia User Agent policy in setting agent type

https://gerrit.wikimedia.org/r/275760

Change 275760 merged by Joal:
Implement the Wikimedia User Agent policy in setting agent type

https://gerrit.wikimedia.org/r/275760

Nuria closed this task as Resolved.Mar 22 2016, 7:15 PM