Page MenuHomePhabricator

Synthesize results for Product Analytics' review of Metrics Platform event types
Closed, ResolvedPublic2 Estimated Story Points

Description

Part of [Epic] Create Metrics Platform API for Submitting Core Interaction Events

Description

Synthesize and document the findings from Product Analytics' assessments of Metrics Platform proposed core interaction event types.

Additional Notes

Coming out of T345729, data scientists from Product Analytics added their assessments of coverage for current instruments using the proposed Metrics Platform core interaction events to a spreadsheet.

Acceptance Criteria

  • Summarize finding/learnings in the spreadsheet and note which production instruments are not well covered by the proposed event types.
  • Make recommendations for what new event types might be needed based on PA reviews.

Event Timeline

  1. The { action, action_subtype, action_source, action_context } 4-tuple that we've implemented (based on @nshahquinn-wmf's suggested in T343331#9137988) covers all of the events that R&DS have listed in the spreadsheet
  2. Establishing common verbs (e.g. init, click), is a valuable step but having concrete schemas for those verbs isn't
    1. Implementing convenience methods for the verbs is cheap
    2. It's not clear how to encourage R&DS/feature teams to review event types with these verbs in mind. For example: @jwang highlighted that a large swath of analytics/mediawiki/ipinfo_interaction event types could be mapped to click events, e.g. action=accept_disclaimer -> action=click,action_subtype=accept,action_source=disclaimer and action=close_disclaimer -> action=click,action_subtype=close,action_source=disclaimer
    3. Having concrete schemas for all of the proposed verbs is possible but inherently limits the types of events that can flow on a stream. Fortunately, we've implemented the new API in such a way that it uses the base schema by default (see #1) but can use an alternative schema if necessary. We should consider creating concrete schemas per verb when the need arises
  3. The following actions don't have proposed verbs yet (but could still be captured – see #1):
SchemaActionData ScientistProposal?
analytics/mediawiki/web_ui_reading_depthpageUnloaded@jwangdeinit? Let's bikeshed this!
analytics/mediawiki/ipinfo_interactioncopy@jwangcopy
analytics/legacy/editattemptstepfirstChange@MNeisler
analytics/legacy/editattemptsteploaded, ready@MNeisleraction=init,action_subtype=loaded, action=init,action_subtype=ready ?
analytics/mediawiki/mediasearch_interactionquickview_filename_copy, quickview_wikitext_link_copy@cchenaction=copy,action_source=…
analytics/mediawiki/structured_task/article/link_suggestion_interactionswipe@nettrom_WMFswipe

@MNeisler, @cchen, and @KCVelaga all highlighted the need for the following verbs: edit, search, and error. Search could be also represented by represented by click or touch but I don't think edit and error could be represented by other verbs. Indeed, edit would be a suitable replacement for the firstChange action in the analytics/legacy/editattemptstep schema.