Page MenuHomePhabricator

[Spike] Understand usage of table of contents on mobile devices
Closed, ResolvedPublic

Description

Description

On mobile web we sort of have a table of contents, in the form of the collapsed section headings. However there are two potential shortcomings of this approach:

  1. You have to scroll (sometimes quite a lot) in order to get to it, because it's below the first section
  2. Once you start exploring a section it's difficult to get back to it

We explored a few potential improvements here. You can view the prototypes (on a mobile device) here.

In order to better understand the priority of potential work in this area we would like to understand usage of table of contents on mobile devices via the Wikipedia Android and iOS apps.

Usage data

  • What percentage of people use the table of contents per session?
  • What percentage of people use the table of contents per page?
  • When the table of contents is used, how many times do people interact with it per page?
  • Do we know anything about the position on the page from which people initially open the table of contents (e.g. from the top of the page, scrolled down a bit)

Android
[data here]

iOS
[data here]

Event Timeline

@JMinor @JTannerWMF can you let us know if you'd be able to gather this data for us, and if so roughly when we could expect it? Thanks : )

Hi I do have some data on this for Android but we don't track TOC that granularly on iOS.

I can collect the last 90 days of data from our MobileWikiAppToCInteraction schema. We don't have data on scroll distance/position,just clicks and interactions and time spent during TOC interactions.

I can put together some data for your use, what is your deadline for this data? Also we need to add Product-Analytics to this ticket for tracking when this request is nailed down.

Hi I do have some data on this for Android but we don't track TOC that granularly on iOS.

I can collect the last 90 days of data from our android_article_toc_interaction schema. We don't have data on scroll distance/position,just clicks and interactions and time spent during TOC interactions.

I can put together some data for your use, what is your deadline for this data? Also we need to add Product-Analytics to this ticket for tracking when this request is nailed down.

Hey @SNowick_WMF, thanks for your reply. Would the end of November be possible as a deadline for you?

SNowick_WMF triaged this task as Medium priority.

Hi @alexhollender_WMF - Yes that deadline works for me, pulling data now. Will post here when ready.

Hi @alexhollender_WMF - I wasn't able to answer all your questions but was able to get some general TOC usage data that will hopefully help with your project.

  • What percentage of people use the table of contents per session?

We do not have session tokens for events in the MobileWikiAppToCInteraction schema so I calculated the percentage of unique users opening TOC by day.

The daily average percent of active Android users who opened the TOC feature in the app was 24.1% with an average 2.05 TOC open events per unique user that interacts with TOC.
90 day event data

  • What percentage of people use the table of contents per page?

I have this data for TOC open events by page (see above) but not by percentage of users, please follow up with me on Slack if you want to clarify this item so we can discuss. I'm not sure how we would measure this without comparing all users page load events vs toc users page events but I can't do that by page since I don't have wiki info for TOC users.

  • When the table of contents is used, how many times do people interact with it per page?

The daily average TOC open event count per unique page is 1.48 with the caveat that unique page_ids can be duplicated across multiple wikis (and my dataset does not specify wiki) so this count should be considered a low estimate.
90 day event data

  • Do we know anything about the position on the page from which people initially open the table of contents (e.g. from the top of the page, scrolled down a bit)

We do not have position data for user interactions with TOC, just counts of section clicks. See data

@SNowick_WMF thanks so much for this data. It's super helpful for us. I will be adding it to the project page for In-page Navigation, and it will help guide our future efforts.

Out of curiosity, what would we need in order to answer the question about the position on the page from which people initially open the table of contents?

Hi @alexhollender_WMF - we would need to add tracking for page positions for TOC clicks which is something you can request for Android engineers to implement. We may also be able to track this through our breadcrumb tracking but that too would need to be added to code to track.