Page MenuHomePhabricator

Backend for wikistats 2.0
Closed, ResolvedPublic0 Story Points

Description

This will be subtasked, but essentially we're implementing the design that was discussed here: https://docs.google.com/document/d/10cTkWcxOE89kx_HejlAbRyiRjlhXL13Cii0hfPOki4c/edit#heading=h.6fww1k1wmrtd

Event Timeline

Nuria created this task.Jan 26 2017, 4:48 PM
Nuria renamed this task from Technical stack for wikistats 2.0 to Backend for wikistats 2.0 .Apr 25 2017, 6:39 PM
bd808 added a subscriber: bd808.May 19 2017, 2:51 PM

I see that this epic has cloud-services-team as a dependency on https://www.mediawiki.org/wiki/Wikimedia_Technology/Goals/2017-18_Q1#Analytics_Engineering. Is there a specific ask of Cloud Services here or just a general need for some support as components are deployed?

Nuria added a comment.May 19 2017, 2:57 PM

Just support, we plan to deploy a storage (druid? clickhouse? other?) to labs with the data from data lake for public access, I think labs team support would mean doing code reviews for puppet and general consulting oh the appropriate way to do things on labs.

bd808 added a comment.May 19 2017, 7:20 PM

@Nuria thanks for the clarification. I think we can handle helping with that. If you don't have hardware budget for deploying the new datastore we'll probably have to scrounge around and see if there is capacity in any of our existing data storage systems. That would be my only concern at this point.

Nuria added a comment.May 20 2017, 4:59 PM

We do have budgetfor 3-druid like hosts. Besides refreshes that is our top priority ask for next year

Milimetric moved this task from Next Up to In Progress on the Analytics-Kanban board.
Milimetric updated the task description. (Show Details)

One version of the history schema simplified and loaded to test how Druid can work as a direct back-end for AQS:

https://pivot.wikimedia.org/#test-wks2-metrics

Milimetric set the point value for this task to 0.
Milimetric moved this task from In Progress to Parent Tasks on the Analytics-Kanban board.
Nuria closed this task as Resolved.Dec 14 2017, 6:33 PM