Page MenuHomePhabricator

Determine current percentage of the mobile site among desktop device pageviews
Closed, ResolvedPublic

Description

Background

Addressing a question from @alexhollender :
Find out what percentage of pageviews from desktop devices (defined as those clients which our Varnish code, according to a user agent regex there, will not redirect from e.g. en.wikipedia.org to en.m.wikipedia.org unless a cookie indicates they opted in) goes to the mobile site (e.g. en.m.wikipedia.org). Data should be broken out by the percentage of the 50 most frequent browser/OS combinations.

IIRC this has been done before by e.g. @dr0ptp4kt, but I would need to look up where exactly and find the current version of that regex.

Acceptance criteria

  • Determine the current percentage of the desktop site among mobile device pageviews
  • Determine the current percentage of the mobile site among desktop device pageviews

Both also limited to views by logged-in users.

It would be ideal if this were delivered in a way such that we can continue to monitor it going forward, rather than a one-time snapshot.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 13 2018, 4:32 AM

Note for later: @dr0ptp4kt pointed me to the location of the regex: https://github.com/wikimedia/puppet/blob/HEAD/modules/varnish/templates/text-frontend.inc.vcl.erb and also to the thread where he had looked at this back in 2016: T131315#2307306 (it was about the converse question, i.e. how often the desktop site is used by mobile devices).

MBinder_WMF triaged this task as Normal priority.Apr 24 2018, 8:11 PM
MBinder_WMF moved this task from Triage to Backlog on the Product-Analytics board.Apr 24 2018, 8:11 PM
ovasileva updated the task description. (Show Details)Aug 19 2018, 12:03 PM

Moving the conversation from T201973#4504915 over here.

At Wikimania we heard from a number of editors that they use "desktop" mode on their mobile devices because the default mobile experience doesn't suit their needs. We want to get a sense for the size of that population, as well as how common this behavior is (for example what % of editors use desktop mode on their phones, and among those who do it what % of their total page views happen in desktop mode?). It might make sense to scope this question to logged in users (as a proxy for editors) since we're not necessarily interested in people who use desktop mode for reading purposes.

@Tbayer pointed out that the question above is related to this task (it is the converse). While that may be the case the application (and urgency) of the information requested remains quite different. This task is meant to help us prioritize bugs that occur when using Minerva from a non-mobile device, whereas the answer to the second question is meant to help us understand the size of the population of editors that rely on desktop mode on mobile devices. @Tbayer does it make sense to represent both requests in this task, or shall I make a separate task for figuring out how common it is for people to use desktop mode on mobile devices?

ovasileva raised the priority of this task from Normal to High.Aug 21 2018, 8:41 AM

Moving the conversation from T201973#4504915 over here.
At Wikimania we heard from a number of editors that they use "desktop" mode on their mobile devices because the default mobile experience doesn't suit their needs. We want to get a sense for the size of that population, as well as how common this behavior is (for example what % of editors use desktop mode on their phones, and among those who do it what % of their total page views happen in desktop mode?). It might make sense to scope this question to logged in users (as a proxy for editors) since we're not necessarily interested in people who use desktop mode for reading purposes.
@Tbayer pointed out that the question above is related to this task (it is the converse). While that may be the case the application (and urgency) of the information requested remains quite different. This task is meant to help us prioritize bugs that occur when using Minerva from a non-mobile device, whereas the answer to the second question is meant to help us understand the size of the population of editors that rely on desktop mode on mobile devices. @Tbayer does it make sense to represent both requests in this task, or shall I make a separate task for figuring out how common it is for people to use desktop mode on mobile devices?

That's a good point. Raising the priority accordingly

Moving the conversation from T201973#4504915 over here.
At Wikimania we heard from a number of editors that they use "desktop" mode on their mobile devices because the default mobile experience doesn't suit their needs. We want to get a sense for the size of that population, as well as how common this behavior is (for example what % of editors use desktop mode on their phones, and among those who do it what % of their total page views happen in desktop mode?). It might make sense to scope this question to logged in users (as a proxy for editors) since we're not necessarily interested in people who use desktop mode for reading purposes.
@Tbayer pointed out that the question above is related to this task (it is the converse). While that may be the case the application (and urgency) of the information requested remains quite different. This task is meant to help us prioritize bugs that occur when using Minerva from a non-mobile device, whereas the answer to the second question is meant to help us understand the size of the population of editors that rely on desktop mode on mobile devices. @Tbayer does it make sense to represent both requests in this task, or shall I make a separate task for figuring out how common it is for people to use desktop mode on mobile devices?

The former - it makes sense to represent both in this task, as @ovasileva had already done on Sunday (T189554#4513048) following our conversation elsewhere.

Thanks for spelling out the rationale here for the more timely part of the request, and yes, as mentioned in last week's email thread, we should now be able to obtain these numbers limited to logged-in users as well - I'll add it to the task description.

Tbayer updated the task description. (Show Details)Aug 21 2018, 4:25 PM

Growth team is interested in this for our "Understanding first day" project. We are splitting traffic into mobile and desktop, but we're doing so using the URL. I made that project's epic a parent of this task to help us follow along. CC @kostajh @nettrom_WMF

kzimmerman removed Tbayer as the assignee of this task.Apr 18 2019, 7:20 PM
kzimmerman added a subscriber: kzimmerman.

@ovasileva and @MMiller_WMF is this needed for your teams?

@ovasileva and @MMiller_WMF is this needed for your teams?

Yes - I think it will help us to establish a benchmark when reporting on Advanced mobile contributions.

@kzimmerman -- I agree that this would be good to at least have a benchmark on. We frequently use the URL as an approximation for desktop/mobile, and each time we remember the caveat that some set of desktop users prefer the mobile site, but we don't really know how much it is. That said, I don't expect the number to be very high, and so I don't think this is critical knowledge for the Growth team.

I was curious about whether this was doable in Superset, and looks like it is to some extent. See Dashboard: https://superset.wikimedia.org/superset/dashboard/34/

This shows the contribution % for the top 4 operating systems (Windows, Android, iOS, and Mac OS X)

And this shows OS Family x Access Method (absolute numbers):

Overall, rates of using mobile site on desktop devices are low (1-2%), and even lower for desktop site on mobile devices (<1%).

@ovasileva @MMiller_WMF I think the above resolves this request, but if not, let me know; this would probably fall under AMC work.

kzimmerman closed this task as Resolved.Jun 26 2019, 2:24 AM
kzimmerman claimed this task.
kzimmerman moved this task from Next Up to Doing on the Product-Analytics board.

Thanks for looking into this, @kzimmerman. This aligns with what I expected. What does "contribution %" mean? Does it mean you're looking only at edits?

@MMiller_WMF This is for all non-bot pageviews. "Contribution %" is just the percentage (e.g. visits to mobile web contribute ~1% of pageviews from Windows OS). (I think I defaulted to the language Superset uses when I wrote that; they call it "Contribution".)