Page MenuHomePhabricator

Schema for EventLogging tracking to new Edit history and Diff screens
Closed, ResolvedPublic

Description

Before we move on to other stuff, I'd like to get the new edit history and diff experiences wired up for basic usage tracking.

Specific questions we should be able to answer:

    • How many users, and how many times per user have they opened the Article History?
    • How many users, and how many times per user have they opened the single revision (from history) screen? Can we break that down by language?
  • How many users, and how many times per user have they started the Comparison flow (ie. selected Compare from the top bar)?
  • How many users, and how many times per user have they completed the Comparison flow (ie. actually opened a "diff" comparison view)?
  • What is the split between anonymous and logged in users for these events? (ie. Do only logged in editors use these feautres, or are they used by the full audience?)
  • How often do users attempt to thank an editor?
  • How often do users successfully thank an editor?

Please see parent epic for context. Most of that epic is complete/shipped, so this would be to track usage of the feature as shipped in v6.5 in late November.

Event Timeline

kzimmerman triaged this task as Medium priority.
kzimmerman moved this task from Triage to Current Quarter on the Product-Analytics board.

@cmadeo It looks like these data points need to be added to code to enable tracking, do you know status on where we are with these?

@SNowick_WMF I don't but I'll ping @Tsevener who might know! Thanks for your help!

Hi @SNowick_WMF - we have not started on this ticket at all, so no events should be firing unless Article History already had some in it from before (though I don't see any code like that with a quick search). I haven't participated in event logging on a new feature yet so I'm a bit hazy on what needs to happen first but I think we need schemas defined from your end before we can start (tagging @JoeWalsh to correct me if I'm wrong). Here's an example ticket with schemas from our Reading Lists feature. https://phabricator.wikimedia.org/T190748 Thanks!

FOR REVIEW @Tsevener @cmadeo

Schema:https://meta.wikimedia.org/wiki/Schema:MobileWikiAppiOSEditHistoryCompare

Documentation for schema: https://docs.google.com/presentation/d/19RiNU6l6pVN7SXFmOCrHCPnJwiWkKW8CJ-9FcDUL9G4/edit#slide=id.p
Please see Slide 8 for questions about additional data to consider collecting if needed.

For breaking down by language, just need to confirm that the default value "wiki" shows the language used while viewing article, ie enwiki = language is English.

The request for reporting on this data should be made separately once we confirm data calls are up and running.

SNowick_WMF raised the priority of this task from Medium to High.Feb 6 2020, 9:52 PM
SNowick_WMF moved this task from Doing to Needs Review on the Product-Analytics (Kanban) board.
SNowick_WMF added a subscriber: RhinosF1.

Feedback from @cmadeo

I do think that it would be helpful to track 'attempts' (esp. around thanking anonymous users).

Also (and apologies if I'm just not seeing/understanding correctly) but are we tracking the clicks into the individual revision view from the diff view? Yes, added an extra screenshot to indicate that measure too>

COMPLETED

  • Added notes on needing 2 specific revision_view points tracked
  • Added thank_fail to schema and docs
SNowick_WMF renamed this task from Add EventLogging tracking to new Edit history and Diff screens to Schema for EventLogging tracking to new Edit history and Diff screens.Feb 18 2020, 6:50 PM

@cmadeo @Tsevener @JMinor Are we ready to close this ticket? When we are ready for next steps we can branch off this or go back to the parent ticket.

SNowick_WMF lowered the priority of this task from High to Medium.Mar 3 2020, 7:07 PM

@SNowick_WMF this should be ready to test in TestFlight build #1731.

Initial test of all events for new schema mobilewikiappiosedithistorycompare,
confirmed events show in logs and in events database:
"action":"show_history"
"action":"revision_view"
"action":"compare1"
"action":"compare2"
"action":"thank_try"
"action":"thank_success"

Pending: Need to verify "action":"thank_fail" which happens when user is not logged in, waiting for tables to populate with data from devices.

Will retest when Beta goes live on App Store.

Test Documentation: https://docs.google.com/spreadsheets/d/1sFjdIdURk1ounRccIH7XKzcOFvUBk7C3zQ8rOjx46Kg/edit?usp=sharing

@SNowick_WMF There are now enough installs of 6.6 that this can be confirmed in prod and resolved.

Reporting for these events is live on Superset - lmk if there are more dimensions to add. Dashboard here