Page MenuHomePhabricator

Estimate Community Updates module experiment sample size
Closed, ResolvedPublic

Description

Context: Slack thread

T360485 is Growth's first experiment that will not be using GrowthExperiments infrastructure that assigns newly registered accounts to an experiment group, but will instead include all editors who visit their Homepage during the experiment and will rely on Metrics Platform's experiment flagging and user bucketing system.

Once the experiment measurement plan is done T365889 and we have a list of which wikis the experiment is going to be run on, we will need to know how many users we expect to be enrolled in the experiment so that we can determine if the backing store is able to handle the number of rows that will be created from the experiment.

A potential lower bound could be the maximum number of unique editors (across those wikis) who have visited their Homepage recently.
A potential upper bound could be the total active editors across those wikis.

Event Timeline

mpopov triaged this task as Medium priority.
mpopov moved this task from Triage to Current Quarter on the Product-Analytics board.

The estimate has been done and the associated googledoc shared with stakeholders for review. In short, we're in need of about 10k visitors to the Homepage, which in a worst-case scenario we're expected to get in 1.5 months. We also have valid reasons to expect that worst-case scenario to not come true, but we've also defined leading indicators to tell us what is happening just in case.

Closing this as resolved.