Page MenuHomePhabricator

Add tagging to webrequest refine process
Closed, ResolvedPublic8 Story Points

Description

As part of the webrefine process we need to calculate the tags ("portal", "wikidata") that will be used to later split the webrequest dataset into smaller sets that are more query-able.

Event Timeline

Nuria created this task.Jul 26 2017, 5:01 PM
Nuria added a comment.Jul 26 2017, 5:23 PM

Alter we need to run:

alter table webrequest add columns (tags array<string> COMMENT 'List containing tags qualifying the request, ex: [portal, wikidata]. Will be used to split webrequest into
smaller subsets.')

Change 367940 had a related patch set uploaded (by Nuria; owner: Nuria):
[analytics/refinery@master] Add tagging as part of webrequest refine process

https://gerrit.wikimedia.org/r/367940

Nuria changed the point value for this task from 0 to 3.Jul 26 2017, 8:09 PM

Tested this code with some fake inserts on 1002, will test bit a bit more data, i just used 1 hour.

Nuria updated the task description. (Show Details)Jul 26 2017, 8:17 PM
Restricted Application added a subscriber: PokestarFan. · View Herald TranscriptJul 26 2017, 8:17 PM
Nuria claimed this task.Jul 27 2017, 2:58 PM
Nuria changed the point value for this task from 3 to 5.
Nuria moved this task from Next Up to In Progress on the Analytics-Kanban board.
Nuria moved this task from In Code Review to Paused on the Analytics-Kanban board.Aug 7 2017, 4:23 PM

Change 367940 merged by Joal:
[analytics/refinery@master] Add tagging as part of webrequest refine process

https://gerrit.wikimedia.org/r/367940

Nuria closed this task as Resolved.Aug 31 2017, 4:30 PM
Nuria changed the point value for this task from 5 to 8.