Hypothesis
If we create task-oriented documentation for analysis with GrowthBook then analysts and other users with elevated access will be able to independently use GrowthBook to analyze experiments conducted with Test Kitchen, eliminating reliance on Test Kitchen Automated Analytics MVP and unblocking broader adoption of GrowthBook.
High-level pieces of the work involved
- On-wiki documentation for how to accomplish various tasks
- Potentially a series of short videos to accommodate different learning styles
- Holding "GrowthBook office hours" for anyone to pop into, separate from my bookable consultation hours
Key results
KR1: Learning resources
Various types of users can learn how to accomplish relevant tasks:
- Everyone
- Access GrowthBook (done as part of SDS2.3.3 Application permissions, cf. T420695)
- Product Managers and Senior Leadership:
- View and understand experiment results
- Note: rely on https://docs.growthbook.io/app/experiment-results as much as possible
- Understand detected SRM for massive-sample experiments
- Software Engineers
- Create an experiment (done as part of SDS2.3.X Documentation, TBD)
- Product Analysts (and Software/Data Engineers)
- Define a new metric (and fact table, as needed)
- Note: rely on https://docs.growthbook.io/app/metrics as much as possible but have WMF-specific documentation, such as the boilerplate stuff we need to include in fact table queries
- Guidelines and recommendations for designing metrics and drafting corresponding instrumentation specifications
- Examples?
without requiring handholding and additional guidance from the Experiment Platform team.
KR2: Widespread adoption
Test Kitchen customers use GrowthBook instead of TK AA (to the extent possible) for automated analysis of their experiments.