Page MenuHomePhabricator

Add regexps that match the bots that follow the User-Agent policy {hawk}
Closed, ResolvedPublic3 Estimated Story Points


Currently, the does not have regular expressions that match the bots that follow the conventions specified in We should implement this, and tag them as "bot" (or "spider" :-), it's a low hanging fruit I think.

Event Timeline

mforns created this task.Feb 3 2016, 11:04 PM
mforns raised the priority of this task from to Needs Triage.
mforns updated the task description. (Show Details)
mforns added a project: Analytics.
mforns added a subscriber: mforns.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptFeb 3 2016, 11:04 PM
jayvdb added a subscriber: jayvdb.Feb 3 2016, 11:14 PM
elukey added a subscriber: elukey.Feb 4 2016, 8:51 AM
Milimetric triaged this task as Medium priority.Feb 4 2016, 6:08 PM
Milimetric moved this task from Incoming to Analytics Query Service on the Analytics board.
Milimetric set Security to None.
Milimetric added a subscriber: Milimetric.

research whether the new expressions that we're adding have a match on the cluster

Nuria edited projects, added Analytics-Kanban; removed Analytics.Feb 25 2016, 5:34 PM
Nuria added a subscriber: Nuria.

Moving this task to kanban so it is up for grabs given that it is pretty small

Milimetric set the point value for this task to 3.Mar 3 2016, 5:38 PM
madhuvishy moved this task from Next Up to In Progress on the Analytics-Kanban board.

Change 275760 had a related patch set uploaded (by Madhuvishy):
Implement the Wikimedia User Agent policy in setting agent type

Change 275760 merged by Joal:
Implement the Wikimedia User Agent policy in setting agent type

Nuria closed this task as Resolved.Mar 22 2016, 7:15 PM