Stream maps cluster requests into HDFS [3pts] {hawk}
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Ironholds
	Aug 11 2015, 6:41 PM

Description

Requests from the maps cluster are now flowing into Kafka and streaming across, but aren't actually going anywhere. The Maps team would like this data to be pushed into Hadoop as a new webrequest_source ("maps", presumably) so that it's possible to analyse Maps requests.

Details

	Project	Branch	Lines +/-	Subject
	analytics/refinery	master	+4 -4	Make camus import webrequest_maps from new maps varnish cluster

Customize query in gerrit

Related Objects

Mentioned In: rANREcb01c882a504: Remove webrequest bits and add webrequest maps

Event Timeline

Ironholds created this task.Aug 11 2015, 6:41 PM

Ironholds raised the priority of this task from to Needs Triage.

Ironholds updated the task description. (Show Details)

Ironholds added a project: Analytics-Backlog.

Ironholds added a subscriber: Ironholds.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 11 2015, 6:41 PM

Change 230535 had a related patch set uploaded (by Yurik):
Make camus import webrequest_maps from new maps varnish cluster

https://gerrit.wikimedia.org/r/230535

gerritbot added a project: Patch-For-Review.Aug 11 2015, 6:42 PM

• Tfinc added a project: Maps-Sprint.Aug 11 2015, 6:46 PM

• Tfinc set Security to None.

• Tfinc moved this task from Backlog to Stalled/Waiting on the Maps-Sprint board.

JAllemandou claimed this task.Aug 12 2015, 2:18 PM

JAllemandou edited projects, added Analytics-Kanban; removed Analytics-Backlog.

JAllemandou moved this task from Next Up to In Progress on the Analytics-Kanban board.

JAllemandou moved this task from In Progress to In Code Review on the Analytics-Kanban board.Aug 12 2015, 2:36 PM

JAllemandou renamed this task from Stream maps cluster requests into HDFS to Stream maps cluster requests into HDFS [3pts] {hawk}.Aug 12 2015, 2:38 PM

JAllemandou mentioned this in rANREcb01c882a504: Remove webrequest bits and add webrequest maps.Aug 13 2015, 10:34 AM

JAllemandou moved this task from In Code Review to Ready to Deploy on the Analytics-Kanban board.Aug 13 2015, 2:36 PM

JAllemandou moved this task from Ready to Deploy to Done on the Analytics-Kanban board.

• kevinator closed this task as Resolved.Aug 14 2015, 3:23 PM

• kevinator added a subscriber: • kevinator.

Yurik moved this task from Stalled/Waiting to Done on the Maps-Sprint board.Aug 14 2015, 11:25 PM

Change 230535 abandoned by Ottomata:
Make camus import webrequest_maps from new maps varnish cluster

Reason:
This was done by joseph in another commit.

https://gerrit.wikimedia.org/r/230535

Yurik added a project: Maps.Nov 7 2015, 7:34 AM

Restricted Application added a project: Discovery-ARCHIVED. · View Herald TranscriptNov 7 2015, 7:34 AM

Restricted Application added a subscriber: StudiesWorld. · View Herald Transcript

Stream maps cluster requests into HDFS [3pts] {hawk}Closed, ResolvedPublicActions

Description

Details

Related Objects

Event Timeline

Stream maps cluster requests into HDFS [3pts] {hawk}
Closed, ResolvedPublic
Actions