Page MenuHomePhabricator

Collect on-going numbers related to TemplateData editor dialog use
Closed, ResolvedPublic5 Estimated Story Points

Description

Collect data on:

  • Number of times the dialog is closed with actual changes being saved vs the dialog being abandoned (also differentiate between whether the user was editing existing TemplateData or adding new TemplateData)
  • Number of times the dialog opened to edit existing TemplateData (track template name)
  • Number of times the dialog opened to add new TemplateData (track template name)

In the TemplateData dialog, track:

  • Change of parameter type from default ‘unknown’ to something specific (track interaction with UI element i.e. count "clicks" on non-default options)
  • Change of priority from default ‘optional’ to another priority (track interaction with UI element i.e. count "clicks" on non-default options)
  • Add or change information to a descriptive, help-oriented parameter field - individually for: label, description, example, default (differentiation between add and change is not necessary)
  • Add or change template description (differentiation between add and change is not necessary)
  • Change ParamOrder (only interested in whether the user did any drag and drop change or no change at all, i.e. do not count clicks)

Segment data by:

  • Segment across the four wikis we are currently tracking: en, de, fa, tr
  • Segment by editor experience count (0-10, 11-100, 101-1.000, 1.001-10.000, 10.001+)
  • No need to segment per template
  • Make sure it's visible how fine-grained each number is. E.g. a user might open the dialog 1 time, but interact with type/priority multiple times.

Other requirements:

  • Add the topic to the inclusion list for sanitization

Note: No information is currently being tracked for the TemplateData editor.

Implementation notes

New EventLogging topic: Schema:TemplateDataEditor

Actions defined:

  • dialog-open-create - Open the dialog to create a new TemplateData record where none existed.
  • dialog-open-edit - Open the dialog to edit an existing TemplateData.
  • parameter-default-change - Edit a parameter's "Default" field.
  • parameter-description-change - Edit a parameter's "Description" field. Sent once per parameter per dialog session.
  • parameter-example-change - Edit a parameter's "Example" field. Sent once per parameter per dialog session.
  • parameter-label-change - Edit a parameter's "Label" field. Sent once per parameter per dialog session.
  • parameter-priority-change - Change a parameter's priority away from the default "Optional". Sent once per parameter per dialog session.
  • parameter-reorder - Drag parameters to change their order. Sent once per dialog session.
  • parameter-type-change - A parameter's "Type" field was changed away from the default "Unknown". Sent once per parameter per dialog session.
  • save-page-create - User attempts to save the page, and we are creating a new TemplateData record.
  • save-page-edit - User attempts to save changes to an existing TemplateData record.
  • template-description-change - Edit the template's Description field. Sent once per dialog session.

We will also collect data on which individual performed an action, to answer questions like "What percentage of TemplateData editors perform action X"?

Event Timeline

Lena_WMDE set the point value for this task to 3.
Lena_WMDE removed the point value for this task.
Lena_WMDE set the point value for this task to 8.
awight moved this task from Doing to Sprint Backlog on the WMDE-QWERTY-Sprint-2020-11-18 board.
awight added a subscriber: awight.
awight moved this task from Sprint Backlog to Doing on the WMDE-QWERTY-Sprint-2020-11-18 board.

The requirements are clear, I'll go ahead with implementation.

Change 644765 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/TemplateData@master] Lint with "commonsjs" rules

https://gerrit.wikimedia.org/r/644765

Change 644766 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/TemplateData@master] Migrate to package files: ext.templateDataGenerator.editPage

https://gerrit.wikimedia.org/r/644766

Lena_WMDE changed the point value for this task from 8 to 5.Dec 2 2020, 10:35 AM

Change 644778 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/TemplateData@master] Migrate to package files: ext.templateDataGenerator.data

https://gerrit.wikimedia.org/r/644778

Should we track when the dialog is opened but JSON data is corrupted, and the user is prompted to replace its content? Probably a vanishingly rare edge, but we don't know yet.

Change 644804 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/TemplateData@master] Higher-level distinction when creating new TemplateData

https://gerrit.wikimedia.org/r/644804

Change 644835 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/TemplateData@master] Log dialog open, distinguish between create and edit

https://gerrit.wikimedia.org/r/644835

Change 644855 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/TemplateData@master] Send metrics when applying changes

https://gerrit.wikimedia.org/r/644855

Change 644859 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/TemplateData@master] Send metrics for template property interactions

https://gerrit.wikimedia.org/r/644859

Change 644765 merged by jenkins-bot:
[mediawiki/extensions/TemplateData@master] Lint with "commonsjs" rules

https://gerrit.wikimedia.org/r/644765

Change 644766 merged by jenkins-bot:
[mediawiki/extensions/TemplateData@master] Migrate to package files: ext.templateDataGenerator.editPage

https://gerrit.wikimedia.org/r/644766

Change 644778 merged by jenkins-bot:
[mediawiki/extensions/TemplateData@master] Migrate to package files: ext.templateDataGenerator.data

https://gerrit.wikimedia.org/r/644778

Change 644804 merged by jenkins-bot:
[mediawiki/extensions/TemplateData@master] Higher-level distinction when creating new TemplateData

https://gerrit.wikimedia.org/r/644804

Change 644835 merged by jenkins-bot:
[mediawiki/extensions/TemplateData@master] Send metrics when opening the dialog

https://gerrit.wikimedia.org/r/644835

Change 644855 merged by jenkins-bot:
[mediawiki/extensions/TemplateData@master] Send metrics when applying changes

https://gerrit.wikimedia.org/r/644855

Change 644859 merged by jenkins-bot:
[mediawiki/extensions/TemplateData@master] Send metrics for template property interactions

https://gerrit.wikimedia.org/r/644859

Change of priority from default ‘optional’ to another priority (track interaction with UI element i.e. count "clicks" on non-default options)

"priority" is a conceptual category for three independent boolean parameter properties in the low-level model: "required", "suggested", and "deprecated". The way I'll implement this requirement is to record an event if (a) all three properties were false, and (b) any checkbox becomes enabled.

Change 646669 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/TemplateData@master] Track parameter property changes

https://gerrit.wikimedia.org/r/646669

Change 646670 had a related patch set uploaded (by Awight; owner: Awight):
[analytics/refinery@master] Sanitize and keep TemplateDataEditor events

https://gerrit.wikimedia.org/r/646670

awight removed awight as the assignee of this task.Dec 8 2020, 8:54 AM

Change 646669 merged by jenkins-bot:
[mediawiki/extensions/TemplateData@master] Track parameter property changes

https://gerrit.wikimedia.org/r/646669

Change 646670 merged by Mforns:
[analytics/refinery@master] Sanitize and keep TemplateDataEditor events

https://gerrit.wikimedia.org/r/646670

Change 655063 had a related patch set uploaded (by Andrew-WMDE; owner: Andrew-WMDE):
[mediawiki/extensions/TemplateData@master] Fix for parameter type change not being logged when type is initially undefined.

https://gerrit.wikimedia.org/r/655063

Change 655063 merged by jenkins-bot:
[mediawiki/extensions/TemplateData@master] The parameter-type-change event should also fire if type was initially undefined

https://gerrit.wikimedia.org/r/655063

Lena_WMDE claimed this task.