We will be tracking user engagement metric in Piwik. Tasks
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • JMinor | T123679 EPIC: As the Product Owner I'd like to know how users are using the main features of the app | |||
| Duplicate | None | T124103 Use cocoapods-keys to remove committed tokens/keys from repo & obfuscate them | |||
| Resolved | • Fjalapeno | T124303 Migrate iOS app to send events to piwik.wikimedia.org | |||
| Declined | None | T123008 As a PM, I can understand how users navigate content so that I can make educated decisions about how to optimize our UI/UX | |||
| Resolved | • JMinor | T126693 As a data analyst, I want to be able to calculate retention using a standard EL event | |||
| Resolved | • Fjalapeno | T126766 Convert existing Piwik events to the new analytics spec |
For item 2 and 3, here's a pass at a high level spec for events using piwik.
Like Google Analytics Piwik uses a four field event strucutre for tracking activity:
[Category][Event][Label][Value]
For our purposes we'll use the four fields to track events using these dimensions
[Context][Action][Content type][Value]
By using the Category as a Context, we can make use of both dimensions answering questions about how a feature is used both within a specific screen, but also its engagement level across the app. Note that Content type and Value are both optional and value in particular will not be used in most cases.
An example from real life™:
Story: "User taps the "Save for later" button on the Featured Article card of the Explore feed."
Event: [explore][save][featured article]
or a more theoretical example, showing use of Value:
Story: "User taps to read the 3rd article in the search results"
[search][tap through][article][search rank=3]
Context- Context refers to the screen or subscreen in which an event occurs. Essentially each screen is a context, and Events' Context field should be populated with the screen name, ideally in line with the English localized name and Spec name for that view.
Action - Actions taken by the user. Actions which span multiple contexts (ie. tapping a preview to load an article) is attributed to the source context (ie. [Explore][tap through][section type]).
Content type - Content type is a generic identification of the type of article the event is occurring on. So, for example, in the feed this refers to the section type (Featured Article, recommendation, etc). As we “fully metric” the app, this could also refer to Image vs. Article for example, or languages selected in the language picker, etc.
Value - not widely used, but a numerical value attached to an event or content. Example would be if we added analytics for search might be [search][tap through][article][search rank] where the search rank of the selected article would be recorded for analysis of how effective our search ranking were.
Piwik registers actions and user activity based on pages, and logs URLs for those pages. Because the app is a mix of internal views and webview pages, this only partially works for tracking Views across the app.
Specifically, the view name should be sent as the page URL. In the case of articles only the domain (en.wikipedia.org) should be sent. We should never send the full article path. This data is both too granular to be immediately useful and adds significant additional privacy concerns.
Ideally in the initial implementation I should be able to see the following in Piwik:
Less prioritized data:
Review existing EL schemas
@JMinor did we determine which schemas are no longer needed for EL?
Would like to know what we can rip out / replace with Piwik
Is there a reason to remove EL stuff? Given the potential scaling issues with Piwik, and lack of operational support, I'd suggest we keep EL in there at least for initial version.
@JMinor some questions / comments
What is Action > Load? What is a load? Is that a preview?
If a load isn't a preview, I'm not quite sure how a preview is logged.
If a user just clicks on a link, do we send 3 events? Load, Impression, Tap through?
It looks like content types aren't mutually exclusive:
Image and Picture of the Day
Article, Recommended Article, Nearby Article, Most Read Article, Main Page, Featured Article
It seems like mostly conflation of the context with the content type? If we want more fine grained contexts, should we break up explore into separate contexts for each section? Or maybe content type isn't the right dimension? We really only have 2 content types, articles and images. Maybe it is helpful to think of it as more of a sub-context?
Anyways - maybe you can walk me through some of this over hangout so I can understand a bit better.
To the issue of content types mixing, lets make content optional and only send when we have specific types that are meaningful.
Discussed on hangout, and edited in spec:
Also clarified that peek previews should not be considered a "Page View" on a new page, but an event on the existing context. When/if the push through occurs to the full article view, a page view on the destination should only be registered at that time.
@JMinor do you want a content type for the language picker (select a different language for an article?)
@JMinor - adding "Disambiguation" context (when viewing similar pages for an article)
I was thinking it might be nice to send the language code they are switching into as the target, if its easy?
Not a requirement for 5.0.0, though as I'm not sure how we'd meaningfully use this anytime soon.
This is the state of analytics following the conversion to the new spec here: T126766
These are implemented:
All tabs
search
articles
Page views are not implemented for anything else (including "more" lists).
Note: Page views are not logged for previewed content (as speced).
Action coverage is mostly centered around explore, article lists, and articles. Below are some descriptions of what is not implemented.
Contexts that aren't implemented (so no actions are logged from these):
Share sheet
Gallery
Settings
ToC
Specific actions that aren't implemented:
Impressions anywhere besides explore
Show/Hide toc
Error
share
switch language
@JMinor do you want to know the date & site for "Most Read" content type & "More Most Read" context? This might help us figure out how diverse the "Most Read" sections are in persons feed and how they interact with them.
@BGerstle-WMF I don't think we need the dates, since we can look at events over time and see if clicks on Most Read decline or show periodicity? Language would be nice, but we're not sending it for Featured Article either, so lets see if we have time before the release to augment that.
@Fjalapeno This looks good. The only things I'd see as additional priorities to get in are share and switch language actions on Article context. Everything else listed is either nice-to-have or not needed for any upcoming specific analysis.