Precursor
Let's get a draft schema set before working on this. It will help guide the spike and find any problems with it.
Background
We would like to run an A/B test on the sticky headers navigation feature to determine whether users are navigating less while reading. We will be looking at the amount of scrolling up (and down) the page with the hypothesis that users in the test group will scroll up less per page.
We would like to know the following:
- total number of pixels scrolled up and down per page
- number of sections opened/closed per page (we could potentially use the section usage schema: https://meta.wikimedia.org/wiki/Schema:MobileWebSectionUsage)
Before building out T197718 we want to determine how to instrument and test this feature.
@ovasileva @Tbayer @alexhollender @phuedx @Jdlrobson met to discuss options. We'll update this card with a summary of that conversation and associated spikes we'll need to carry out to do so.
We do not intend to run multiple A/B tests simultaneously.
Acceptance criteria
- Determine the best way to instrument the feature so that we can collect the data and answer the questions outlined above
- Write a task with the proposed implementation
Questions to answer
- Which bits are simple
- Which bits would be complicated/give rise to concern.
- Any problems with the draft schema?
- The height is variable depending on what is open and closed.
Notes
- scroll event is triggered by user interactions, internal links (e.g. #Heading) and opening overlays (notably clicking images opens the media viewer overlay which triggers a scroll to top). @phuedx pointed out that looking at event.currentTarget.srcElement will allow us to distinguish scrollbar-triggered (srcElement === document) and UI element triggered scrolls.
- We need to decide whether to send multiple events per page or follow an approach similar to https://meta.wikimedia.org/wiki/Schema:MobileWikiAppPageScroll