Load edit history data into Druid
Closed, ResolvedPublic0 Estimated Story Points
Actions

Assigned To

None

Authored By

	• Nuria
	Apr 4 2016, 6:09 PM

Related Objects
Search...

Status	Assigned	Task
Resolved	None	T120037 Vital Signs: Please provide an "all languages" de-duplicated stream for the Community/Content groups of metrics
Resolved	None	T120036 Vital Signs: Please make the data for enwiki and other big wikis less sad, and not just be missing for most days
Resolved	odimitrijevic	T130256 Wikistats 2.0.
Resolved	None	T131779 Implement Pages Created & Count of Edits full vertical slice
Resolved	None	T131786 Load edit history data into Druid
Duplicate	None	T141472 Research spike: load enwiki data into Druid to study lookup table performance
Declined	JAllemandou	T134792 Spike - Slowly Changing Dimensions on Druid
Resolved	JAllemandou	T141473 Productionize loading of edit data into Druid (contingent on success of research spike)

Event Timeline

• Nuria created this task.Apr 4 2016, 6:09 PM

JAllemandou moved this task from Incoming to Backlog (Later) on the Analytics board.Apr 11 2016, 4:37 PM

Milimetric moved this task from Backlog (Later) to Dashiki on the Analytics board.Jun 2 2016, 5:11 PM

• Nuria edited projects, added Analytics-Kanban; removed Analytics.Jul 4 2016, 4:48 PM

• Nuria moved this task from Next Up to Parent Tasks on the Analytics-Kanban board.Jul 20 2016, 3:05 PM

• Nuria renamed this task from Load into Druid when data is ready to Load edit history data into Druid when data is ready for enwiki .Jul 27 2016, 7:12 PM

• Nuria updated the task description. (Show Details)

• Nuria renamed this task from Load edit history data into Druid when data is ready for enwiki to Load edit history data into Druid .Jul 27 2016, 7:14 PM

• Nuria updated the task description. (Show Details)

• Nuria created subtask T141472: Research spike: load enwiki data into Druid to study lookup table performance.

• Nuria created subtask T141473: Productionize loading of edit data into Druid (contingent on success of research spike).

• Nuria merged a task: T138269: solving issues with scalability , querying and high cardinality dimensions (maybe hosting in Druid, clickhouse or other).Jul 27 2016, 7:16 PM

• Nuria added subscribers: nshahquinn-wmf, Akeron, Zppix.

Seems that we should be able to close this parent task this quarter right?

• Nuria closed this task as Resolved.Mar 20 2017, 3:31 PM

• Nuria reopened this task as Open.

• Nuria closed subtask T141473: Productionize loading of edit data into Druid (contingent on success of research spike) as Resolved.Mar 22 2017, 7:47 PM

Ping @joal, are there any tasks remaing regarding druid and edit data?

@Nuria: Currently checking the code works, but we have everything for 2 years data loaded and regenerated every month.