Page MenuHomePhabricator

Add ismobile attribute to X-Analytics header
Closed, ResolvedPublic

Description

In order to make T389696: Analyze impact for webrequest and unique devices pipelines to derive access_method without m-dot domain possible, we need something in the webrequest logs that the refinery can use to make its decision by.

Assumptions:

  • T214998 will not change the fact that the X-Subdomain: m request header is set internally between Varnish and MediaWiki.
  • T214998 will be implemented by applying the X-Subdomain: m header on requests from mobile user agents directly on the canonical domain (today we first redirecting to m-dot, and then add the header there).
  • Most request headers, including this one, are logged by varnishkafka or HAProxy as part of events in the webrequest topic.

Ideas:

  1. In Varnish, when we extend the X-Analytics header with additional key-value pairs, add one for ismobile=1 at the same time that we set X-Subdomain: m.
  2. Or – In MediaWiki, when MobileFrontend decides to render a mobile web page (i.e. because it sees X-Subdomain: m, or useformat=mobile, or something else), add ismobile=1 to X-Analytics header via hook from the XAnalytics extension.

Other ideas? Preferences?

Event Timeline

Krinkle updated the task description. (Show Details)

Other ideas? Preferences?

IIRC Varnish is the decision maker in production – MobileFrontend simply responds to the presence of the X-Subdomain header. For that reason, my preference would be (1).

Just my 0.02 fundamental monetary units.

If this code is in MobileFrontend, it won't detect mobile hits to non-MF wikis like Wikifunctions, so that would push towards (1) as well.

IIRC Varnish is the decision maker in production – MobileFrontend simply responds to the presence of the X-Subdomain header. For that reason, my preference would be (1).

Varnish does not decide what format is used. In shouldDisplayMobileViewInternal(), MobileFrontend decides whether to use the mobile format in response to two query parameters, two cookies, X-Subdomain and mobile device detection. For example, https://en.wikipedia.org/wiki/Ian_Carmichael?useformat=mobile is the mobile website and https://en.m.wikipedia.org/wiki/Ian_Carmichael?useformat=desktop is the desktop website. If the idea is to classify requests by format, I think the best way to do that is with a response header from MobileFrontend.

If the idea is to classify requests by device type, well, both MF and Varnish have code for that. X-Subdomain is currently strictly derived from the subdomain, it's not a device type header.

Why not add a device type request header, and vary on that? In cluster_fe_recv, before mobile_redirect, do device detection and set say X-Device-Type: mobile. Have mobile_redirect consult that header. Have MF also consult that header by reconfiguring $wgMFMobileHeader. Why keep X-Subdomain when we're dropping subdomains?

Krinkle triaged this task as High priority.

In a sense the question is whether access_method should classisy the client, or the server response.

  • client - Classify whether the client advertises itself as a "mobile" device, or
  • server - Classify whether we served a "mobile" formatted version of the page.

Today, we mostly do "client", because Varnish decides whether to redirect traffic to m-dot, and the refinery then uses .m. to compute access_method. This means for edge cases like en.m.wikipedia?useformat=desktop and en.wikipedia?useformat=mobile we classify them today based on the hostname, not based on the served format. On the other hand, for cookies we do it the other way around. I doubt this was intentional, but indeed if you opt-in to the m-dot cookie from a desktop browser, or opt-out from mobile on a mobile device, we happen to classify your chosen experence, ignoring the user-agent, because these cookies redirect you to/from mobile (unlike the query parameter). So we're not consistent today here.

There are two more reasons for Option 1:

  • Today we compute access_method in the wmf.webrequest dataset regardless of is_pageview. This means we can analyze traffic patterns, to not just for pageviews, but also for thumbnails on upload.wikimedia.org, and API requests to /w/api.php and /w/rest.php, using access_method as a way to group by access method. I don't know how important these use cases are, but, by opting to set this in Varnish, we keep this largely the same as today and thus don't need to consult/migrate these queries, thus leading to a quicker decision. If there is merit to change this, I think we can do that a separate task outside the critical path of T389696: Analyze impact for webrequest and unique devices pipelines to derive access_method without m-dot domain. For example, we could end up with a hybrid where MobileFrontend sets it on pageviews to 1 or 0 and leave it to Varnish to classify the rest.
  • Jon has voiced a desire to phase out MobileFrontend within the coming year. And, as James points out, Wikifunctions already doesn't have MF installed. In a future where MobileFrontend doesn't exist, and pageviews use a responsive skin without server-side variations, there would no longer be a server-side place to set ismobile=1 (it'd be expensive to vary the CDN on this, just to add a single header from MediaWiki PHP). In that future, we'd almost certainly revert back to wanting to classify the client, as there'd no longer be a server-side format difference.

Change #1160381 had a related patch set uploaded (by Krinkle; author: Krinkle):

[operations/puppet@production] varnish: Set X-Analytics `ismobile=1` for mobile requests

https://gerrit.wikimedia.org/r/1160381

Mentioned in SAL (#wikimedia-operations) [2025-06-20T19:10:03Z] <sukhe> sudo cumin 'A:cp' "disable-puppet 'merging CR rOPUP11603816fccd'": T390924

Change #1160381 merged by Ssingh:

[operations/puppet@production] varnish: Set X-Analytics `ismobile=1` for mobile requests

https://gerrit.wikimedia.org/r/1160381

Mentioned in SAL (#wikimedia-operations) [2025-06-20T19:16:08Z] <sukhe> enabling puppet on cp4037 to merge CR rOPUP11603816fccd: add `ismobile=1' for mobile requests: T390924

Mentioned in SAL (#wikimedia-operations) [2025-06-20T19:19:05Z] <sukhe> sudo cumin -b11 'A:cp' "run-puppet-agent 'merging CR rOPUP11603816fccd'": T390924

Mentioned in SAL (#wikimedia-operations) [2025-06-20T19:19:22Z] <sukhe> sudo cumin -b11 'A:cp' "run-puppet-agent --enable 'merging CR rOPUP11603816fccd'": T390924

This is now live.

$ curl -I 'https://en.m.wikipedia.org/wiki/Main_Page' -H 'X-Wikimedia-Debug: 1'

x-analytics: …;https=1;ismobile=1;debug=1;…