Page MenuHomePhabricator

Context items: enhanced instrumentation post-deployment data checks
Closed, ResolvedPublic

Description

Overview

This task involves the work with making sure we are logging data in such a way that we will be able to answer the questions* listed in the "Metric checks" section below.

*These questions come from T221252

Instrumentation spec

VE feature instrumentation spec

Metric checks

These are the questions we would like to be able to answer:

  • 1. What percent of contributors “open” an Edit Card?
    • 1.1 What kind of Edit Card (i.e. link, citation, image, template, etc.) is it?
  • 2. What percent of contributors “open” a context item?
    • 2.1What kind of context item (i.e. link, citation, image, template, etc.) is it?
  • 3.What percent of contributors take an action within an Edit Card? Asked another way: What percentage of contributors who are shown an Edit Card tap “edit” or “remove”?
    • 3.1 What kind of Edit Card is it? Link, citation, image, template, etc.
  • 4. What percent of contributors take an action within a context item? Asked another way: What percentage of contributors who are shown a context item tap “edit” or “remove”?
    • 4.1 What kind of context item is it? Link, citation, image, template, etc.
  • 5. What percent of contributors tap edit within a context item and then abort the subsequent dialog without making any changes?
    • 5.1 What kind of Edit Card is it? Link, citation, image, template, etc.
  • 6. What percent of contributors tap edit within an Edit Card and then abort the subsequent dialog without making any changes?
    • 6.1 What kind of Edit Card is it? Link, citation, image, template,
  • 7. What percent of contributors take an action within a context item, make changes in the subsequent dialog and then abort their edit?
    • 7.1 What kind of Context Item is it? Link, citation, image, template, etc.
  • 8.What percent of contributors take an action within an Edit Card, make changes in the subsequent dialog and then discard their changes?
    • 8.1 What kind of Edit Card is it? Link, citation, image, template, etc.
  • 9.What percent of contributors tap edit within a context item, make changes in the subsequent dialog and then apply their changes?
    • 9.1 What kind of context item is it? Link, citation, image, template, etc.
  • 10. What percent of contributors tap edit within an Edit Card, make changes in the subsequent dialog and then apply their
    • 10.1 What kind of Edit Card is it? Link, citation, image, template, etc.
  • 11. What percent of contributors tap edit within a context item, make changes in the subsequent dialog and then publish those changes?
    • 11.1 What kind of context item is it? Link, citation, image, template, etc.
  • 12. What percent of contributors tap edit within an Edit Card, make changes in the subsequent dialog and then publish those changes?
    • 12.1 What kind of Edit Card is it? Link, citation, image, template, etc.

"Done"

  • Confirm gathering data for 2 weeks will give us enough volume to create a meaningful baseline
  • We have confirmed we are logging the data necessary to answer the questions listed in the "Metric checks" section above

Event Timeline

ppelberg created this task.Apr 17 2019, 3:45 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 17 2019, 3:45 PM
ppelberg renamed this task from Edit card: confirm instrumentation for existing VE linking flow to Context items: enhanced instrumentation post-deployment data checks.Jul 5 2019, 4:44 PM
ppelberg reassigned this task from ppelberg to Neil_P._Quinn_WMF.
ppelberg added a project: Product-Analytics.
ppelberg updated the task description. (Show Details)
ppelberg updated the task description. (Show Details)Jul 12 2019, 7:58 PM
Neil_P._Quinn_WMF triaged this task as Medium priority.Jul 15 2019, 1:14 PM
Neil_P._Quinn_WMF moved this task from Triage to Next Up on the Product-Analytics board.
Neil_P._Quinn_WMF closed this task as Resolved.Jul 30 2019, 1:01 PM
Neil_P._Quinn_WMF raised the priority of this task from Medium to High.
Neil_P._Quinn_WMF moved this task from Next Up to Doing on the Product-Analytics board.
Neil_P._Quinn_WMF added a subscriber: DLynch.

Everything looks good based on the checks I did and discussions with @DLynch. My full notebook is on GitHub.