Background

An experiment can have two or more variations, e.g. control and A, control and A, B, and C, etc.

We want to be able to answer the following questions about an experiment and its variations:

• Which variation had the most conversions?
  • e.g. Which variation led to more clicks on the Donate button?
  • e.g. Which variation led increased the median user session length?
  • e.g. Which variation led to the user clicking the top result in the search autocomplete?

Proposal

In order to answer the above, we propose the following event shape:

interface BaseExperimentConversion {
  timestamp: string;
  userToken: string;
  experimentID: string;
  experimentFriendlyName: string;
  experimentVersion: string;
  experimentUpdated: string;
  variationID: string;
  variationFriendlyName: string;
  conversionType: 'binomial'
}

type CountExperimentConversion {
  conversionType: 'count';
  conversionCount: number;
}

type DurationExperimentConversion {
  conversionType: 'duration';
  conversionDuration: number;
}

type ExperimentConversion = BaseExperimentConversion & CountExperimentConversion;

TODO

• The proposal is validated
• We determine whether an event shape (or a partial event shape) like in the proposal can be exported to GrowthBook
• (Optional) A JSONSchema Event Platform schema is created