Page MenuHomePhabricator

EPIC: Productionizing Wikimetrics {dove}
Closed, ResolvedPublic

Description

Moving parts of Wikimetrics onto production systems
Benefits:

  • The Data Warehouse is on the production DBs. Wikimetrics will be able to run its Vital Signs queries against this DW.
  • Labs DB (which wikimetrics currently uses) has had many outages and issues lately
  • Production DBs perform better

Drawbacks:

  • Wikimetrics was originally envisionned to run on public data (data in labs db)
  • Will buy us two instances. One in labs, one in production.
  • Pressure to move data to public cannot be made that easy any longer

Scope:

  • Only productionizing backend (queue, scheduler for recurrent reports. 3 instances (dev, staging, production))
  • Not the web frontend.

Reasons why wikimetrics has not been pushed into production:

  • python dependencies:

Yuvi in the meantime debianized some and also offered to help with the remaining ones

Tasks:

  • T76769 [Dev - 8 pts] upgrade wikimetrics to trusty (using same install process as we have now. Probably just setting up new instances, not uprgrading them)
  • T76789 (Yuvi + Ops) debianizing any (scheduler/queue) wikimetrics dependencies that are not in trusty / our apt
  • T76790 changing puppetization for scheduler/queue (Kevin -> assign to Andrew and provide context, add blocking task)
    • T76791 Puppet Production role class for wikimetrics scheduler/queue (Kevin -> merge into 76790)
  • T76770 [Dev 13 pts] change wikimetrics in labs to use debian packages, try in staging 1st, make sure this works alongside of pip install with webserver part too
  • T76779 [Dev 13 pts] Fix oauth and do a quick pre-security review.
  • T76780 [Dev 8 pts] Change admin script so it doesn't have to make symlinks, configure production instance to write files to /a/limn-public-data
  • T76792 [Ops] new group that would allow you to sudo to the wikimetrics user (to enable dev to run admin script)
  • T76782 [Security] have security review

Event Timeline

ggellerman raised the priority of this task from to Needs Triage.
ggellerman updated the task description. (Show Details)
ggellerman changed Security from none to None.
ggellerman added a subscriber: ggellerman.
kevinator triaged this task as Medium priority.Dec 4 2014, 3:45 PM
kevinator moved this task from Incoming to Backlog on the Analytics-Engineering board.
kevinator added a subscriber: kevinator.
kevinator updated the task description. (Show Details)Feb 4 2015, 4:47 PM
kevinator renamed this task from EPIC: Productionizing Wikimetrics to EPIC: Productionizing Wikimetrics {dove}.Feb 4 2015, 5:41 PM
kevinator lowered the priority of this task from Medium to Low.
kevinator edited projects, added Analytics-Kanban; removed Analytics-Engineering.
Restricted Application added a project: Analytics. · View Herald TranscriptNov 21 2017, 10:56 PM
Reedy changed the status of subtask T76782: security review of Wikimetrics from Open to Stalled.Sep 11 2018, 7:44 PM