Page MenuHomePhabricator

Desktop behavior data requests
Closed, ResolvedPublic

Assigned To
Authored By
alexhollender_WMF
Aug 6 2019, 1:16 PM
Referenced Files
F30318394: proportion_sidebar_views.png
Sep 13 2019, 5:17 PM
F30316364: proportion_header_views_loggedout.png
Sep 13 2019, 5:17 PM
F30316583: proportion_header_views_loggedin.png
Sep 13 2019, 5:17 PM
F29975829: image.png
Aug 6 2019, 2:03 PM
F29975795: image.png
Aug 6 2019, 2:03 PM
F29975848: image.png
Aug 6 2019, 2:03 PM
F29975821: image.png
Aug 6 2019, 2:03 PM
F29975782: image.png
Aug 6 2019, 2:03 PM

Description

Description

We would like to begin to develop an understanding of the usage/behavior of various elements on the desktop site as we start to think about improvements we can make to the desktop experience.

1. Main menu (sidebar)
Data questions: what percentage of users use links in the sidebar? What is the usage for all of the various links?
Audiences: logged-in users, logged-out users
Why this is relevant: we're curious about a more modular design, wherein the main menu is collapsible. It could even act different for different audiences. Knowing how often it gets used, and by whom, will inform our thinking.
Notes:

  • some editors customize their main menus
  • some links in the main menu are available elsewhere - we are particularly curious about usage of the links as located in the main menu
    image.png (900×1 px, 580 KB)

2. Search
Data questions: what percentage of users use search? How many searches do users of search perform per session?
Audiences: logged-in users, logged-out users
Why this is relevant: currently search is a relatively small element on the page, and is only accessible from the top of the page (i.e. you can't perform a search if you're scrolled down, reading an article). We're curious if people have difficulty finding search, or perhaps just don't think of searching the site because they don't notice it.
Notes:

image.png (762×1 px, 728 KB)

3. Language links
Data questions: what percentage of users use language links?
Audiences: logged-in users, logged-out users
Why this is relevant: currently you have to scroll in order to find the language links. We're wondering if a more prominent location, perhaps closer to the article context itself, would make it easier for users to find.
Notes:

image.png (663×1 px, 494 KB)

4. User links in header
Data questions: what percentage of users use the user links in the header? What is the usage for all of the various links?
Audiences: logged-in users, logged-out users
Why this is relevant: currently the user links take up quite a bit of space in the header. As we think of ways to tidy up the site it would be helpful to understand what the usage is like there.
Notes:

image.png (434×1 px, 350 KB)

5. Table of contents
Data questions: what percentage of users use the table of contents? For users who do use it, how many table of contents links do they click per article?
Audiences: logged-in users, logged-out users
Why this is relevant: currently it is not possible to access the table of contents without being near the top of an article. It seems like a key navigational component for reading, possibly deserving of some kind of UX that allows people to access it regardless of which part of the page they're scrolled to.
Notes:

  • on iOS and Android the table of contents is always accessible. It could be interesting to compare usage there vs. web
    image.png (765×1 px, 366 KB)

Event Timeline

kzimmerman triaged this task as Medium priority.Aug 6 2019, 9:21 PM
kzimmerman moved this task from Triage to Next Up on the Product-Analytics board.
kzimmerman added subscribers: ovasileva, kzimmerman.

Megan may not be able to get to this before Wikimania next week. If not, it sounds like you'll need this in about a month, @ovasileva @alexhollender?

Megan may not be able to get to this before Wikimania next week. If not, it sounds like you'll need this in about a month, @ovasileva @alexhollender?

Sounds good. And that's correct - having some of this by wikimania would be nice to have, but not crucial. Afterwards, we'll have some time before we need the entirety of the data.

Sorry for the delay! Below is a summary of the results. Link to full analysis. My ability to answer some of the questions was limited based on available data and instrumentation but I tried to substitute other options where possible.

@ovasileva @alexhollender - Let me know if you have any questions.

1. Main menu (sidebar)
Data Notes:

  • Used sample data from webrequest and pageview_hourly table.
  • Data is from July 2019 and includes desktop users on en.wiki.
  • Results are based on views to the pages linked in the sidebar from an internal referrer (within the wiki project). Clicks directly to links in the sidebar are not currently tracked and, as a result, some of these views may include clicks from other locations on the website.

What percentage of users use links in the sidebar?
Only about 0.5% of all logged out users and 1.6% of all logged-in desktop users clicked on one of the pages linked in the sidebar. Note this is an approximation based by grouping by the client_ip and user_agent.

What is the overall usage?
The main page is by far the most viewed sidebar linked page on desktop (about 68% of all user views to a sidebar linked page are to the main page) followed by the what_links_here, current_events, and download as pdf pages. This is for both logged in and logged out users.

proportion_sidebar_views.png (1×2 px, 69 KB)

2. Search

Data Notes: Data from searchsatisfaction table and webrequest. Includes data from August 2019 and for desktop users across all projects. Search events include both full text and autocomplete searches.

What percentage of users use search?
About 1.4% of logged out desktop users and about 2.5% of logged in desktop users use search. Note that user counts are an approximation based on a grouping of client ip and user agent.

How many searches do users of search perform per search session??
There is an average of 6.8 searches per search session a day on desktop. This is for both logged in and logged out users.

3. Language Links

Data notes: Data from wmf.interlaguage_navigation. Includes data for all projects and from July 2019.

What percentage of users use language links?
Unable to show the percent of users based on current instrumentation. Instead, I've provided the % of all pageviews requests for each project instead.

About 0.38% of all pageview requests from desktop are clicks to language links. Here's a break down for some of the top sized wikis. The percent_of_pageview_requests column shows the percent of all pageview requests that are clicks to the language links for each project.

projectpercent_of_pageview_requests
fr.wikipedia0.385
de.wikipedia0.371
it.wikipedia0.315
ja.wikipedia0.274
zh.wikipedia0.238
en.wikipedia0.218
es.wikipedia0.202
ru.wikipedia0.182

Smaller wikis have a higher percent of clicks to the language links.

projectpercent_interlanguage_navigation
kg.wikipedia10.363773
cv.wikipedia9.953629
gd.wikipedia9.470999
pa.wikipedia9.375662

5. User Links in Header

Data Notes: Sample data from webrequest and pageview_hourly table for en.wiki. Data is from July 2019 and for desktop users on en.wiki.

What percentage of users use the user links in the header?
About 0.04% of logged-out users and 7% of logged-in users click on of the user page links found in the headers.

What is the usage for all of the various links?
Desktop pageviews from header links for logged in users

The user page, user talk page, and contributions pages are the most viewed pages in the header for logged in users. The user page accounts for about 33% of all desktop pageviews to links in a header for logged in users.

proportion_header_views_loggedin.png (1×2 px, 47 KB)

Desktop pageviews from header links for logged out users.

The create account link is the most viewed page in the header for logged out users. It accounts for about 52% of all desktop pageviews to links in a header for logged out users. Note: There were very few views to then UserLogin page compared to the other pages in the header bar, which seems unlikely. I spent a little time digging into the data but have not yet determined if this is real behavior or a bug.

proportion_header_views_loggedout.png (1×2 px, 45 KB)

5. Table of Contents

Clicks to the table of contents on desktop are not sent to the server (in browser only) so unfortunately I don't believe there's any way to track this currently using available data. We would need to add instrumentation to track those clicks.

@MNeisler follow up from our discussion: can we figure out whether the language links (as a group) get clicked more than the other links in the sidebar? It would be great to look at this per-wiki for maybe 5–6 different wikis (fr, de, it, kg, cv, gd)?

Also, @ovasileva what are your thoughts on the table of contents stuff? It would be awesome if we could understand how that is used but I'm not sure how costly adding instrumentation is.

In T229926#5527195, @alexhollender wrote:

@MNeisler follow up from our discussion: can we figure out whether the language links (as a group) get clicked more than the other links in the sidebar? It would be great to look at this per-wiki for maybe 5–6 different wikis (fr, de, it, kg, cv, gd)?

@alexhollender @ovasileva
Following up on this request. I compared the desktop usage of language links to other links in the sidebar on several large and small wikis.

The tables below show the percent of all sidebar link clicks on desktop by users in June 2019 for languages as a group, the main page, and all other sidebar links. For example, clicks to the language links represent about 39.1% of all sidebar links clicks on English Wikipedia.

In summary, the language links as a group are clicked more than the other links in the sidebar for all wikis reviewed on desktop. Clicks to the main page account for the second most viewed sidebar link. On the larger sized wikis reviewed, clicks to language links represent around half (39-52%) of all clicks to links in the sidebar. On the smaller-sized wikis reviewed, language link clicks represent an even larger portion (77-84%) of all clicks to links in the sidebar.

English Wikipedia

Sidebar link# viewsPercent of all sidebar views
languages546730539.1%
main_page519982937.2%
all other sidebar links332177723.7%

French Wikipedia

Sidebar link# viewsPercent of all sidebar views
languages78439952.9%
main_page42253628.5%
all other sidebar links27526718.6%

German Wikipedia

Sidebar link# viewsPercent of all sidebar views
languages152815249.3%
main_page114607537.0%
all other sidebar links42370313.7%

Italian Wikipedia

Sidebar link# viewsPercent of all sidebar views
languages42808250.8%
main_page24092228.6%
all other sidebar links17309620.6%

Welsh Wikipedia

Sidebar link# viewsPercent of all sidebar views
languages4001182.1%
main_page31966.6%
all other sidebar links553411.3%

Sicilian Wikipedia

Sidebar link# viewsPercent of all sidebar views
languages2562784.2%
main_page24027.9%
all other sidebar links23997.9%

Pashto Wikipedia

Sidebar link# viewsPercent of all sidebar views
languages1031777.0%
main_page181513.5%
all other sidebar links12749.5%

Further details available in the repo and notebook.

Marking this task as resolved for now but let me know if you have any questions. For any follow-up desktop usage data requests, I'd recommend creating a new task and tagging Product-Analytics so the team can triage and address while I'm away (Jan 6-April 14th).