Create UDFs for categorising referers
Closed, ResolvedPublic5 Story Points

Description

We know we'll want to identify external traffic from search engines by their referers. Build a system of UDFs capable of distinguishing:

  1. Traffic from search engines;
  2. Traffic from other sources;
  3. Traffic with no referers.

Furthermore, break (1) down by search engine.

Ironholds updated the task description. (Show Details)
Ironholds raised the priority of this task from to Needs Triage.
Ironholds added a subscriber: Ironholds.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 19 2015, 8:43 PM
Ironholds set Security to None.Oct 19 2015, 8:43 PM
Ironholds edited a custom field.
Ironholds moved this task from Backlog to In progress on the Discovery-Analysis (Current work) board.
Ironholds claimed this task.

Change 247601 had a related patch set uploaded (by OliverKeyes):
[WIP] functions for identifying search engines as referers.

https://gerrit.wikimedia.org/r/247601

Deskana triaged this task as Normal priority.Nov 10 2015, 9:07 PM
Deskana added a subscriber: Deskana.

Moving this to the backlog to more accurately represent its status in the team; it's stalled on us and can't be prioritised more highly right now.

Deskana added a subscriber: mpopov.Nov 24 2015, 9:05 PM
This comment was removed by Deskana.
mpopov claimed this task.Dec 10 2015, 5:37 PM
mpopov moved this task from Backlog to In progress on the Discovery-Analysis (Current work) board.
mpopov removed mpopov as the assignee of this task.Dec 11 2015, 4:09 PM
mpopov added a comment.EditedDec 23 2015, 10:19 PM

I have made changes in response to Joal and Nuria's comments, and have tested the latest version on the cluster. It appears to work. I have reached out to Nuria to see if there are additional tests she had in mind. Otherwise we're on track to getting this done and deployed.

Update: "As long as we test on cluster i think we are good. Have in mind we will not be deploying this probably until January though as half of the team is on vacation until then." - Nuria

Nuria moved this task from Next Up to In Code Review on the Analytics-Kanban board.

Change 247601 merged by Nuria:
Functions for identifying search engines as referers.

https://gerrit.wikimedia.org/r/247601

Nuria closed this task as Resolved.Feb 2 2016, 4:39 PM