Page MenuHomePhabricator

Create instrumentation spec for establishing baseline for interface customization
Closed, ResolvedPublic

Description

Background

The WE2.1 KR states: Ensure a quality reading experience for all users by adapting the default experience for 15% of pageviews, based on the individual needs and constraints of the user. We would like to establish a baseline for this metric by looking at at the frequency of how the interface currently adapts

Acceptance criteria

Evaluate which of the following metrics is currently instrumented:

  • % of pageviews which have set a non-default font size in the Minerva skin
  • % of pageviews which do not use page previews in the Vector 2022 skin (or on desktop overall)
  • % of pageviews which do not use the default width option (and view the site at full width)

Create instrumentation specification for the remaining metrics

Next steps in data process (bolding indicates step we are currently on):

PM and Data Analyst coordinate to identify experiment design, research questions, metrics and guardrails for the new feature and associated metrics.
Data analyst creates a list of events that need to be tracked to calculate each metric
Engineers review the event list and comment on whether we’re already tracking with existing instruments and if new instrumentation is required. New tickets created for any new instrumentation needed.
Data analyst works with engineers to figure out what the new instrumentation should look like and where to store the events.
Data analyst documents all of the above in the instrumentation spec
PM signs off on the plan
Legal & Security sign off on the plan

Instrumentation Spec

Documentation: Instrumentation Spec

Event Timeline

ovasileva updated the task description. (Show Details)
ovasileva moved this task from Incoming to Analyst Consultation on the Web-Team-Backlog board.

@jwang & @ovasileva: Just a quick reminder that you will need to submit the measurement plan / instrumentation specification for Legal & Security review through L3SC. Fortunately it doesn't look like you'll be need to collect any sensitive data about the users so the review should be relatively fast.

Delivery

  • Propose "Event to be tracked" in the instrumentation spec based on PM's question list.
  • Propose schema field name and value for each event, and documented in instrumentation spec.
  • Filled in the schemas and field names to store the events based on the discussion with engineers and PM.

See the baseline collection 1, 2, 3 in Instrumentation Spec

@jwang & @ovasileva: Just a quick reminder that you will need to submit the measurement plan / instrumentation specification for Legal & Security review through L3SC. Fortunately it doesn't look like you'll be need to collect any sensitive data about the users so the review should be relatively fast.

Legal review request now submitted.

@jwang & @ovasileva: Just a quick reminder that you will need to submit the measurement plan / instrumentation specification for Legal & Security review through L3SC. Fortunately it doesn't look like you'll be need to collect any sensitive data about the users so the review should be relatively fast.

Legal review request now submitted.

We have now passed legal review. Resolving this ticket.