Background

The goal of the sticky header is to make functionality usually found at the top of the page more easily accessible from everywhere within the page. We would like to measure this through getting a comparison of the clickthrough of the links at the top of the page vs the sticky header, as well as by measuring whether it decreases the need to scroll to the top of the page

Questions

1. Does the sticky header decrease the need to stroll to the top of the page?
2. Is the functionality added to the sticky header used and how frequently?
   1. What is the clickthrough rate (per pageview or per session) of each item on the sticky header (Note: Maybe overall number of clicks can answer the question)
   2. What is the ratio of clicks of sticky header items to the corresponding items at the top of the page
   3. Is there an overall increase in clicks to the functionality available in the sticky header (i.e. the sum of the sticky header functionality and the corresponding items at the top of the page)
   4. What is the effect of the sticky header introduction on edit rates for editors of varying experience levels (by edit bucket)

Rough plan

• Add new events to https://meta.wikimedia.org/wiki/Schema:DesktopWebUIActionsTracking that correspond to links within the sticky header
• Create a new schema that instruments scrolls to the top of the page (potentially by measuring the viewport and triggering an event beyond a certain threshold)
• Perform an A/B test with the test group seeing the sticky header and the control group seeing the current experience

Next steps in data process (bolding indicates step we are currently on):

• Data analyst creates a list of events that need to be tracked to calculate each metric
• Engineers review the event list and comment on whether we’re already tracking with existing instruments and if new instrumentation is required. New tickets created for any new instrumentation needed.
• Data analyst works with engineers to figure out what the new instrumentation should look like and where to store the events.
• Data analyst documents all of the above in the instrumentation spec
• PM signs off on the plan

Documentation

instrumentation spec for sticky header