This is a parent task for the work to be done for the Better Use of Data Program, which was started in FY2018/19.
# Roadmap FY2019-2020
##### Q1 (July - September)
##### Data Engineering
[] **Event Platform Client Libraries**
[] Develop Event Platform Client specification T228177
[] Develop Event Platform Client test suite
[] Develop Android client T228179
[] Develop iOS client T228180
[] Develop JS browser client T228181
[] Document usage guidelines on-wiki
[] Event Stream & Schema Documentation: MEP Deployment Processes T228656
[] Explore how we may port events to new schema
[] **Cross-platform datasets**
[] Research and architect "unique devices" dataset
[] Research and architect "session length" dataset (see https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Traffic/SessionLength)
##### Data Access
[] Automated dashboard for Product Core Metrics (Readers & Contributors)
[] Internal production release of edits_hourly and editors_monthly Druid datasets (for use in Superset and Turnilo)
##### Data Training
[] Start product team trainings: best practices for working with data in the product development lifecycle
[] Start product team trainings for core metrics: data exploration and reports
##### Tracking
- MEP stream configuration service planning
- MEP schema registry deployment
- Client-side error logging working group
---
##### Q2 (October - December)
##### Data Engineering
[] **MEP for Product**
[] Develop schema registry UI
[] Develop stream configuration service UI
[] Develop CI and commit hooks
[] Develop stream configuration service metadata
[] Cross-platform client-side error logging T229442
[] Evaluate analytics systems capacity
[] Evaluate incremental back-off / rate limiting solutions
[] **Cross-platform datasets**
[] Prototype and test "unique devices" dataset
[] Prototype and test "session length" dataset
[] Recommendation for porting old schemas to new system
##### Data Training
[] Continue product team trainings: best practices for working with data in the product development lifecycle
[] Continue product team trainings for core metrics: data exploration and reports
##### Tracking
- MEP engineering sync
- Client-side error logging working group
---
##### Q3 (January - March)
##### Data Engineering
[] **Common Instrumentation Development Kit (final production form)**
[] Develop production version of Sampling Controller
[] Develop production version of Association Controller
[] **Complex analytics guidance**
[] Document A/B testing procedures
[] Document funnel analysis procedures
[] **Cross-platform datasets**
[] Production tracking for "unique devices" dataset is included in Common Instrumentation Development Kit
[] Production tracking for "session length" dataset is included in Common Instrumentation Development Kit
##### Tracking
- TBD
---
##### Q4 (April - June)
##### Data Engineering
[] Advise all newly-created schema use EventGate-style JSONSchema
[] Port select EventLogging schema to EventGate-style JSONSchema
[] Evaluate feasibility of cross-schema joins
[] Develop automated ingestion pipeline and dashboard defaults
##### Tracking
- TBD