Background
The goal of the new table of contents is to make ToC navigation usually found at the top of the page more easily accessible from everywhere within the page, as well as to provide context on the the article throughout the reading experience, from the location where a reader is reading to larger context on the topic and individual sections.
We would like to measure the effects of introducing the change and the success of the goals stated above quantitatively.
Questions
- Is the new table of contents is used more frequently than the previous table of contents
- Does the new table of contents reduce the need to scroll back to the top of the page
- Does the new table of contents decrease the time people spend scrolling/scrolling quickly (if possible)
- How does the new table of contents affect the time spent on a page
Data instrumentation process step (bolding indicates current step)
- PM and Data Analyst coordinate to identify research questions, hypotheses, and guardrails for the new feature and associated metrics.
- Data analyst creates a list of events that need to be tracked to calculate each metric
- Engineers review the event list and comment on whether we’re already tracking with existing instruments and if new instrumentation is required. New tickets created for any new instrumentation needed.
- Data analyst works with engineers to figure out what the new instrumentation should look like and where to store the events.
- Data analyst documents all of the above in the instrumentation spec
- PM signs off on the plan