For the Revise Tone A/B test we want to better understand how the new Onboarding Quiz performs.
Things we like to understand:
- which users complete the onboarding quiz?
- which questions are answered right/wrong how often?
- how often do people click on which "learn more" link?
- at what question (beyond the first) do people click the "skip all"-button?
- (maybe if possible) how long do people spend on each question?
- overall, what is the distribution of correctly answered questions of finished quizzes?
Except for the first item above which can only be handled via instrumentation, all the other items should be recorded both in instrumentation and Prometheus/Grafana performance metrics. Though, creating the related panels on Grafana is not part of this task.
Acceptance Criteria
- analytics events are being collected to answer the above questions
- performance metrics are being tracked to answer the above questions (except the first)
- we do not intend to collect any IP addresses or Browser User Agent
Notes
- This task also includes creating figuring out how the XLab integration / stream-config / analytics-schema would look like.
- This should use the same schema as T405177: Revise Tone: Instrumentation
- The main steps that have to be instrumented with xLab/Test-kitchen are:
- impression of the onboarding opening ("start of the funnel")
- clicks on "skip all" ("early funnel exit")
- clicks on "Get Started" ("finished the funnel")