Page MenuHomePhabricator

Enhance instrumentation for context items and inspectors
Closed, ResolvedPublicJul 9 2019

Description

This task involves the work with enhancing instrumentation in the "pre-Edit Card" state of VE so we can establish a baseline to measure the impact of Edit Cards and the changes to dialogs/inspectors.

Measurements

Below are the the measurement questions we would like to be able to answer in order to compare contributors' engagement with and use of context items and inspectors before and after the release of Edit Cards v2: T225834.

These measurement questions are sourced from the the "Measurements" section of our Measurement Plan/Edit Cards doc:

  • 1. What percent of contributors “open” an Edit Card?
    • 1.1 What kind of Edit Card (i.e. link, citation, image, template, etc.) is it?
  • 2. What percent of contributors “open” a context item?
    • 2.1What kind of context item (i.e. link, citation, image, template, etc.) is it?
  • 3.What percent of contributors take an action within an Edit Card? Asked another way: What percentage of contributors who are shown an Edit Card tap “edit” or “remove”?
    • 3.1 What kind of Edit Card is it? Link, citation, image, template, etc.
  • 4. What percent of contributors take an action within a context item? Asked another way: What percentage of contributors who are shown a context item tap “edit” or “remove”?
    • 4.1 What kind of context item is it? Link, citation, image, template, etc.
  • 5. What percent of contributors tap edit within a context item and then abort the subsequent dialog without making any changes?
    • 5.1 What kind of Edit Card is it? Link, citation, image, template, etc.
  • 6. What percent of contributors tap edit within an Edit Card and then abort the subsequent dialog without making any changes?
    • 6.1 What kind of Edit Card is it? Link, citation, image, template,
  • 7. What percent of contributors take an action within a context item, make changes in the subsequent dialog and then abort their edit?
    • 7.1 What kind of Context Item is it? Link, citation, image, template, etc.
  • 8.What percent of contributors take an action within an Edit Card, make changes in the subsequent dialog and then discard their changes?
    • 8.1 What kind of Edit Card is it? Link, citation, image, template, etc.
  • 9.What percent of contributors tap edit within a context item, make changes in the subsequent dialog and then apply their changes?
    • 9.1 What kind of context item is it? Link, citation, image, template, etc.
  • 10. What percent of contributors tap edit within an Edit Card, make changes in the subsequent dialog and then apply their
    • 10.1 What kind of Edit Card is it? Link, citation, image, template, etc.
  • 11. What percent of contributors tap edit within a context item, make changes in the subsequent dialog and then publish those changes?
  • 11.1 What kind of context item is it? Link, citation, image, template, etc.
  • 12. What percent of contributors tap edit within an Edit Card, make changes in the subsequent dialog and then publish those changes?
    • 12.1 What kind of Edit Card is it? Link, citation, image, template, etc.

"Done"

  • The instrumentation required for us to be able to answer the measurement questions listed in in the "Measurements" section of our Measurement Plan/Edit Cards doc have been implemented.

Event Timeline

Change 518757 had a related patch set uploaded (by DLynch; owner: DLynch):
[VisualEditor/VisualEditor@master] Enhanced instrumentation for context items and inspectors

https://gerrit.wikimedia.org/r/518757

Change 518757 merged by jenkins-bot:
[VisualEditor/VisualEditor@master] Enhanced instrumentation for context items and inspectors

https://gerrit.wikimedia.org/r/518757

Change 519466 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/extensions/VisualEditor@master] Update VE core submodule to master (f93b871a6)

https://gerrit.wikimedia.org/r/519466

Change 519466 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Update VE core submodule to master (f93b871a6)

https://gerrit.wikimedia.org/r/519466

@matmarex are you working on this task? It is assigned to Peter.

ppelberg renamed this task from Edit card: implement workflow instrumentation to Enhance instrumentation for context items and inspectors.Jul 5 2019, 4:33 PM
ppelberg reassigned this task from ppelberg to DLynch.
ppelberg updated the task description. (Show Details)
ppelberg set Due Date to Jul 9 2019, 7:00 AM.
Restricted Application changed the subtype of this task from "Task" to "Deadline". · View Herald TranscriptJul 5 2019, 4:34 PM

I know we want this on tomorrow's train, is this in code review? It says on Phab its In Progress? @DLynch

I think the generic part is done – some of the logging has to be implemented individually for each inspector/dialog, and they have separate tasks:

(I suppose that for QA, we should ask Neil to verify that the data being logged actually allows checking the things we want, after all changes are done.)

I think the generic part is done – some of the logging has to be implemented individually for each inspector/dialog, and they have separate tasks:

Awesome, @matmarex. Thank you for updating the status of the check on this task – T221252 – and for linking the related tasks.

A point of clarification that I should've added to the task earlier: On 5 July, @Neil_P._Quinn_WMF and @DLynch confirmed the proper instrumentation was in place for us to be able to answer the measurement questions listed in the task description.

cc @JTannerWMF

(I suppose that for QA, we should ask Neil to verify that the data being logged actually allows checking the things we want, after all changes are done.)

Precisely, @matmarex. Here is the ticket for that work: T221254: Context items: enhanced instrumentation post-deployment data checks

Update: this patch will be deployed tomorrow, 11-July.

Hey @DLynch and @matmarex, marcella mentioned this patch is deployed, can we move it to @ppelberg column for close out?

I guess that since we have that separate ticket for the QA (T221254), we might as well push this to product owner review.