Page MenuHomePhabricator

Better Use of Data
Open, NormalPublic

Description

This is a parent task for the work to be done for the Better Use of Data Program, which was started in FY2018/19.

Roadmap FY2019-2020

Q1 (July - September)
Data Engineering
  • Event Platform Client Libraries
    • Develop Event Platform Client specification T228177
    • Develop Event Platform Client test suite
    • Develop Android client T228179
    • Develop iOS client T228180
    • Develop JS browser client T228181
    • Document usage guidelines on-wiki
      • Event Stream & Schema Documentation: MEP Deployment Processes T228656
      • Explore how we may port events to new schema
  • Cross-platform datasets
Data Access
  • Automated dashboard for Product Core Metrics (Readers & Contributors)
  • Internal production release of edits_hourly and editors_monthly Druid datasets (for use in Superset and Turnilo)
Data Training
  • Start product team trainings: best practices for working with data in the product development lifecycle
  • Start product team trainings for core metrics: data exploration and reports
Tracking
  • MEP stream configuration service planning
  • MEP schema registry deployment
  • Client-side error logging working group

Q2 (October - December)
Data Engineering
  • MEP for Product
    • Develop schema registry UI
    • Develop stream configuration service UI
    • Develop CI and commit hooks
    • Develop stream configuration service metadata
  • Cross-platform client-side error logging T229442
  • Evaluate analytics systems capacity
  • Evaluate incremental back-off / rate limiting solutions
  • Cross-platform datasets
    • Prototype and test "unique devices" dataset
    • Prototype and test "session length" dataset
  • Recommendation for porting old schemas to new system
Data Training
  • Continue product team trainings: best practices for working with data in the product development lifecycle
  • Continue product team trainings for core metrics: data exploration and reports
Tracking
  • MEP engineering sync
  • Client-side error logging working group

Q3 (January - March)
Data Engineering
  • Common Instrumentation Development Kit (final production form)
    • Develop production version of Sampling Controller
    • Develop production version of Association Controller
  • Complex analytics guidance
    • Document A/B testing procedures
    • Document funnel analysis procedures
  • Cross-platform datasets
    • Production tracking for "unique devices" dataset is included in Common Instrumentation Development Kit
    • Production tracking for "session length" dataset is included in Common Instrumentation Development Kit
Tracking
  • TBD

Q4 (April - June)
Data Engineering
  • Advise all newly-created schema use EventGate-style JSONSchema
  • Port select EventLogging schema to EventGate-style JSONSchema
  • Evaluate feasibility of cross-schema joins
  • Develop automated ingestion pipeline and dashboard defaults
Tracking
  • TBD

Related Objects

Event Timeline

kzimmerman updated the task description. (Show Details)Jun 6 2019, 5:31 PM
kzimmerman updated the task description. (Show Details)
kzimmerman updated the task description. (Show Details)
jlinehan updated the task description. (Show Details)Jun 6 2019, 6:34 PM
jlinehan updated the task description. (Show Details)
jlinehan updated the task description. (Show Details)Jun 6 2019, 6:46 PM
jlinehan updated the task description. (Show Details)
jlinehan updated the task description. (Show Details)Jun 6 2019, 8:22 PM
jlinehan updated the task description. (Show Details)
jlinehan updated the task description. (Show Details)Jun 6 2019, 8:26 PM
kzimmerman updated the task description. (Show Details)Jun 7 2019, 4:47 AM
kzimmerman updated the task description. (Show Details)Jun 8 2019, 12:34 AM
This comment was removed by Aklapper.
kzimmerman moved this task from Triage to Backlog on the Product-Analytics board.Jun 19 2019, 10:42 PM
phuedx added a subscriber: phuedx.Jun 20 2019, 2:04 PM
kzimmerman updated the task description. (Show Details)Jun 20 2019, 8:49 PM
kzimmerman updated the task description. (Show Details)Jun 21 2019, 11:18 PM
kzimmerman triaged this task as Normal priority.Jun 26 2019, 12:46 AM
kzimmerman updated the task description. (Show Details)Jul 8 2019, 8:27 PM
jlinehan updated the task description. (Show Details)Jul 16 2019, 2:32 PM
jlinehan moved this task from Tracking to To Do on the Better Use Of Data board.Jul 16 2019, 5:15 PM
jlinehan moved this task from To Do to Tracking on the Better Use Of Data board.
kzimmerman updated the task description. (Show Details)Thu, Aug 15, 8:43 PM
kzimmerman updated the task description. (Show Details)Thu, Aug 15, 9:29 PM
jlinehan updated the task description. (Show Details)Thu, Aug 15, 10:00 PM