Productionize reporting of app session metrics {hawk} [8pts]
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• kevinator
	May 2 2015, 1:10 AM

Description

make script take parameters using options parsing libs
make script to append data to an output location accessible to PMs [1]
schedule weekly oozie job

[1] https://hue.wikimedia.org/filebrowser/view/wmf/data/archive/mobile_apps/uniques/

Details

	Subject	Repo	Branch	Lines +/-
	Add oozie job to schedule mobile app session metrics spark job.	analytics/refinery	master	+307 -0
	Productionize app session metrics	analytics/refinery/source	master	+296 -241

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	• madhuvishy	T97876 Productionize reporting of app session metrics {hawk} [8pts]
Resolved	• Nuria	T86535 Mobile PMs has reports on session-related metrics from Wikipedia Apps {hawk}
Resolved	JAllemandou	T91793 Add user_agent map to refined tables {hawk}
Resolved	JAllemandou	T89396 Add x_analytics map to refined tables {hawk}
Resolved	Ottomata	T94596 Make oozie work with spark jobs that use HiveContext

Event Timeline

• kevinator created this task.May 2 2015, 1:10 AM

• kevinator renamed this task from Productionize reporting of app session metrics to Productionize reporting of app session metrics {hawk}.

• kevinator assigned this task to • Nuria.

• kevinator raised the priority of this task from to High.

• kevinator updated the task description. (Show Details)

• kevinator added projects: Patch-For-Review, Analytics-Kanban, Analytics-Clusters.

• kevinator removed a project: Patch-For-Review.

• kevinator set Security to None.

• kevinator removed a parent task: T86535: Mobile PMs has reports on session-related metrics from Wikipedia Apps {hawk}.

• kevinator added a subtask: T86535: Mobile PMs has reports on session-related metrics from Wikipedia Apps {hawk}.

• kevinator added subscribers: • kevinator, • Nuria, Aklapper.

• kevinator removed • Nuria as the assignee of this task.May 2 2015, 1:14 AM

• kevinator mentioned this in T86535: Mobile PMs has reports on session-related metrics from Wikipedia Apps {hawk}.

• kevinator closed subtask T86535: Mobile PMs has reports on session-related metrics from Wikipedia Apps {hawk} as Resolved.

• kevinator added a subscriber: JAllemandou.

Liuxinyu970226 subscribed.May 2 2015, 8:30 AM

• kevinator updated the task description. (Show Details)May 4 2015, 5:07 PM

• kevinator renamed this task from Productionize reporting of app session metrics {hawk} to Productionize reporting of app session metrics [8pts] {hawk}.May 4 2015, 5:10 PM

• kevinator moved this task from Next Up to Tasked_Hidden on the Analytics-Kanban board.

• madhuvishy moved this task from Tasked_Hidden to In Progress on the Analytics-Kanban board.May 18 2015, 3:33 PM

• madhuvishy claimed this task.May 18 2015, 4:07 PM

Change 212573 had a related patch set uploaded (by Madhuvishy):
[WIP] Productionize app session metrics - Parse args using scopt

https://gerrit.wikimedia.org/r/212573

gerritbot added a project: Patch-For-Review.May 21 2015, 5:10 PM

This was blocked on a bug using hive-context in spark.
However, there is a workaround doing something similar to what is here: https://gerrit.wikimedia.org/r/#/c/212541/2/refinery-job/src/main/scala/org/wikimedia/analytics/refinery/job/PageviewAggregator.scala

Milimetric renamed this task from Productionize reporting of app session metrics [8pts] {hawk} to Productionize reporting of app session metrics {hawk} [8pts].Jun 3 2015, 8:56 PM

Change 216009 had a related patch set uploaded (by Madhuvishy):
[WIP] Add oozie job to schedule mobile app session metrics spark job. See also - https://gerrit.wikimedia.org/r/#/c/212573/

https://gerrit.wikimedia.org/r/216009

• madhuvishy moved this task from In Progress to In Code Review on the Analytics-Kanban board.Jun 11 2015, 5:50 AM

• madhuvishy added a subscriber: • JKatzWMF.Jun 12 2015, 8:49 PM

Change 212573 merged by Joal:
Productionize app session metrics

https://gerrit.wikimedia.org/r/212573

• madhuvishy mentioned this in rANRS7db96a09cfbc: Productionize app session metrics.Jun 16 2015, 7:06 PM

Change 216009 merged by Joal:
Add oozie job to schedule mobile app session metrics spark job.

https://gerrit.wikimedia.org/r/216009

• madhuvishy moved this task from In Code Review to Ready to Deploy on the Analytics-Kanban board.Jun 22 2015, 7:32 PM

• madhuvishy mentioned this in rANREe1e91e1fb354: Add oozie job to schedule mobile app session metrics spark job..Jun 22 2015, 7:40 PM

• madhuvishy mentioned this in rANREd1064bc1f6af: Add driver memory as a configurable property to Spark job.Jun 26 2015, 5:47 PM

• madhuvishy moved this task from Ready to Deploy to Done on the Analytics-Kanban board.Jun 26 2015, 8:41 PM