Page MenuHomePhabricator

Extend the MobileWikiAppFeed schema to measure basic engagement metrics
Closed, ResolvedPublic2 Estimated Story Points

Description

This task is about extending the MobileWikiAppFeed schema by various events, in order to obtain a set of metrics about the usage of the Explore feed that were identified as required earlier.

  • Add an entered event, to be sent when the user enters the feed (by opening the app, going back to the feed from an article, ...)
    • Yields the % of daily users who use the feed feature
    • Yields the refresh rate, when combined with refresh events
  • Add an exited event, to be sent when the user leaves the feed (by leaving the app, clicking through to an article, etc.)
    • Yields the bounce rate
    • Yields the average minutes spent on each Explore view (together with the entered event)
    • Yields the total time spent on the feed during a given timespan (per user)
    • Might help estimate the % of time spent (per user) on the feed
  • Add a shown event, sent with cardType value once a card is displayed
    • Yields the # of card impressions
    • Should yield clickthrough rate per card type (which cards are the most popular when shown?), if combined with the data from the LinkPreview schema
  • Add a linktapped event, sent with cardType value once a content link in a card is tapped (i.e. an article link in most cases; in case of the "Picture of the Day" cards, the image link; and in the case of "In the news" cards, the link to the news story excerpt from Template:In the news)
    • Yields the clickthrough rate per card type directly (instead of the likely brittle combination with the LinkPreview schema)
    • Yields clicks on “Picture of the Day” (which the Preview schema doesn’t seem to count)
    • Would allow to answer questions on how often (per explore session) users click links
    • First step to measuring “Time spent after click”
  • Add a cardType value for the new announcement cards (T140012)

See also the notes here about what data can and can't be obtained from the current version of the schema.

Event Timeline

Hi @Tbayer. Please associate at least one project with this task to allow others to find this task when searching in the corresponding project(s). Thanks!

How do we log taps on news stories?

That would be a linktapped event sent with the cardType value corresponding to "In the news" cards. (BTW, the schema page currently does not document these values directly, but only refers to "the card type enumeration defined in our code" - a link would be nice.)

The description of this task says:

Add a linktapped event, sent with cardType value once an article link in a card is tapped

This seems to imply that this would only be logged when an article linked from a news story is tapped, not the story itself. For news the funnel is a bit longer: feed, story, link (linktapped)

The description of this task says:

Add a linktapped event, sent with cardType value once an article link in a card is tapped

This seems to imply that this would only be logged when an article linked from a news story is tapped, not the story itself. For news the funnel is a bit longer: feed, story, link (linktapped)

Ah OK, I thought that by "news stories" you meant the "In the news" card itself, not the linked extract from https://en.wikipedia.org/wiki/Template:In_the_news .

In that case, I guess the answer is that these clicks further down the funnel, on the story page, are currently not instrumented as part of this schema. Do you have suggestions on how to integrate this? (Assuming the clickthroughs to the actual articles about the news event are something we want to measure here.)

For now I'll update the task description regarding the linktapped event to more precisely indicate that "article link" is really meant as the content link from the card, besides the already mentioned exception of POTD.

There has been some deliberation with @Dbrant about the precise definition of card impressions, i.e. the exact moment when a shown event should be triggered. I'm suggesting to take the moment when it becomes possible to tap on the card's content link(s) (i.e. to cause a linktapped event). But if that is too difficult to implement, we could also go with another definition. The most important thing is that it should be consistent and well-documented.

BTW the iOS equivalent of the recently added subtask for the new announcement cards is T151836: Track announcement usage in Piwik.

Public update for others who might be following along here: I understand from @Dbrant that most of the work on this is already done, but that right now the task is still blocked on implementation and testing of the shown event (the most challenging one to get right across devices etc.) - which however should be completed fairly soon too.

Change 342765 had a related patch set uploaded (by Dbrant):
[apps/android/wikipedia] Prepare feed coordinator for improved event logging.

https://gerrit.wikimedia.org/r/342765

Change 342766 had a related patch set uploaded (by Dbrant):
[apps/android/wikipedia] Supercharge eventlogging for the feed.

https://gerrit.wikimedia.org/r/342766

Change 342765 merged by jenkins-bot:
[apps/android/wikipedia] Prepare feed coordinator for improved event logging.

https://gerrit.wikimedia.org/r/342765

Change 342766 merged by jenkins-bot:
[apps/android/wikipedia] Supercharge eventlogging for the feed.

https://gerrit.wikimedia.org/r/342766