Page MenuHomePhabricator

Stream maps cluster requests into HDFS [3pts] {hawk}
Closed, ResolvedPublic

Description

Requests from the maps cluster are now flowing into Kafka and streaming across, but aren't actually going anywhere. The Maps team would like this data to be pushed into Hadoop as a new webrequest_source ("maps", presumably) so that it's possible to analyse Maps requests.

Event Timeline

Ironholds raised the priority of this task from to Needs Triage.
Ironholds updated the task description. (Show Details)
Ironholds added a project: Analytics-Backlog.
Ironholds added a subscriber: Ironholds.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 11 2015, 6:41 PM

Change 230535 had a related patch set uploaded (by Yurik):
Make camus import webrequest_maps from new maps varnish cluster

https://gerrit.wikimedia.org/r/230535

Tfinc set Security to None.
Tfinc moved this task from Backlog to Stalled/Waiting on the Maps-Sprint board.
JAllemandou edited projects, added Analytics-Kanban; removed Analytics-Backlog.
JAllemandou moved this task from Next Up to In Progress on the Analytics-Kanban board.
JAllemandou renamed this task from Stream maps cluster requests into HDFS to Stream maps cluster requests into HDFS [3pts] {hawk}.Aug 12 2015, 2:38 PM
JAllemandou moved this task from Ready to Deploy to Done on the Analytics-Kanban board.
kevinator closed this task as Resolved.Aug 14 2015, 3:23 PM
kevinator added a subscriber: kevinator.
Yurik moved this task from Stalled/Waiting to Done on the Maps-Sprint board.Aug 14 2015, 11:25 PM

Change 230535 abandoned by Ottomata:
Make camus import webrequest_maps from new maps varnish cluster

Reason:
This was done by joseph in another commit.

https://gerrit.wikimedia.org/r/230535

Yurik added a project: Maps.Nov 7 2015, 7:34 AM
Restricted Application added a project: Discovery. · View Herald TranscriptNov 7 2015, 7:34 AM
Restricted Application added a subscriber: StudiesWorld. · View Herald Transcript