Page MenuHomePhabricator

RFC: Remove .m. subdomain, serve mobile and desktop variants through the same URL
Open, MediumPublic

Assigned To
None
Authored By
tstarling
Jan 31 2019, 12:19 AM
Referenced Files
None
Tokens
"Love" token, awarded by Rexogamer."Like" token, awarded by Dreamy_Jazz."Love" token, awarded by Iniquity."Love" token, awarded by Trizek-WMF."Like" token, awarded by Ladsgroup."100" token, awarded by Sj."Love" token, awarded by MusikAnimal."Love" token, awarded by Ltrlg."Like" token, awarded by taavi."Like" token, awarded by Mobiledesktop."Barnstar" token, awarded by Gloin_Oin."Like" token, awarded by Njk."Like" token, awarded by Darylgolden."Like" token, awarded by Benjaminikuta."Like" token, awarded by Darxus."Like" token, awarded by Jdforrester-WMF."Love" token, awarded by dbarratt."Dislike" token, awarded by TerraCodes."Like" token, awarded by Platonides."Love" token, awarded by Kaartic."Orange Medal" token, awarded by Addshore."Love" token, awarded by Legoktm."Orange Medal" token, awarded by Krinkle."Mountain of Wealth" token, awarded by Krenair."Love" token, awarded by Harej.

Description

  • Affected components: Wikimedia site configuration.
  • Engineer for implementing the change: Core Platform Team (WMF) and SRE ServiceOps (WMF).
  • Code steward: (idem).

Motivation

The m. subdomain, as in en.m.wikipedia.org, is annoying. Links shared on social media are randomly mobile or non-mobile, and so desktop users often accidentally visit the mobile site. Having separate domains complicates analytics and search engine optimisation.

The m. subdomain does not appear to reflect best current practice. Many websites are able to detect mobile clients and deliver the correct variant through an unmodified URL. For example, WikiTravel does this, using CloudFront and MediaWiki/MobileFrontend.

Requirements
  1. When browsing Wikipedia from mobile web or desktop, and sharing the browser address with someone, the address should produce the default Wikipedia experience for the receiver's end.

Exploration

The first requirement means we either have to redirect in both directions, or vary the server response without a redirect. The latter seems simple and also more performant for the end-user by not having a significant portion of page views delayed by an additional blocking/synchronous client-server roundtrip over the network.

I propose removing the m. subdomain. The frontend would detect the desired device variant based on UA and request cookies, and send it to MediaWiki in a single header, and would vary the resulting cache based on this request header. MediaWiki (MobileFrontend, or core after T100402 is fixed) would use this request header to select the skin.

@BBlack suggests starting with a soft launch. We would initially remove the device detection redirect from en.wikipedia.org to en.m.wikipedia.org. We would redirect away from the old domains only when the traffic on them has declined. Obviously, to avoid breaking links, the redirect will have to be retained forever.

We could look at how device detection works in third-party CDNs for ideas on what to call the request header or any other unresolved details.


Known issues / extra work avoided:

Related Objects

Mentioned In
T363695: Create a Wikimedia login domain that can be served by any wiki
T343283: When I come on a mobile device, it should automatically take me to m.wikifunctions.org
T312042: After log in on mobile Beta Commons, user gets redirected to non-mobile page (and is not logged in there)
T289016: SecurePoll: Mobile redirection on votewiki gives "not logged in" error
T270603: Module site.styles generates different output depending on mobile cookie, if $wgMFSiteStylesRenderBlocking = true;
T264678: UrlShortener should use desktop link has primary
T225814: m.wikidata: login status not detected
T171398: On mobile domain, interwiki links for WMF wikis should be resolved as mobile rather than desktop
T206497: Enable $wgMFNoindexPages for: Italian, Dutch, Korean, Arabic, Chinese, and Hindi Wikipedias
T52399: {{fullurl:}} and {{SERVER}} when used on mobile do not resolve to commons.m.wikimedia.org but to commons.wikimedia.org
T211559: Notification links redirects to desktop site
T195494: Handle mobile domains in core
T198969: Ensure links on the mobile version of pages are not to the desktop version
T60425: Mobile site does not automatically redirect to desktop version (and not possible to use browser "use desktop view")
Mentioned Here
T349408: Use responsive Vector-2022 instead of Minerva for Wikifunctions Mobile and drop the secondary domain/MobileFrontend part
T152882: Many misc wikis lack mobile domains
T156847: Core should be aware of the domain it is running on and render mobile domains where necessary
T195494: Handle mobile domains in core
T353300: Update mobile domain derivation code to match new canonical version
T225814: m.wikidata: login status not detected
T74186: Varnish: Mobile site redirect interferes with OAuth authorization process
T171398: On mobile domain, interwiki links for WMF wikis should be resolved as mobile rather than desktop
T172581: [EPIC] Set up mechanism for archiving Google Search Console data
T215071: Merge Wikipedia subdomains into one, to discourage censorship
T32618: Desktop version fine on mobile UNTIL clicking "disable mobile view"
T44480: Disabling desktop view is broken
T48473: desktop view on mobile domain
T60425: Mobile site does not automatically redirect to desktop version (and not possible to use browser "use desktop view")
T100402: Core should have support for mobile detection and customization

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

@tstarling, this problem:

The m. subdomain, as in en.m.wikipedia.org, is annoying. Links shared on social media are randomly mobile or non-mobile, and so desktop users often accidentally visit the mobile site.

can be easily resolved by detecting desktop/mobile client by User-Agent and 302 redirecting by default desktop clients from mobile version of site to desktop version of site. Now exists redirection only from desktop version to mobile version of site for mobile clients (detected by User-Agent).

I have standalone wiki based on MediaWiki engine and already create such logic using nginx web server and njs script. If you or someone else interested - I can share my njs code and nginx wiki config at github.

main idea of my code looks like this:

// ...
var arg_mobileaction = r.args.mobileaction;
var redirect_host;
if (arg_mobileaction === 'toggle_view_desktop') {
    redirect_host = desktop_host;
} else if( arg_mobileaction === 'toggle_view_mobile' ) {
    redirect_host = mobile_host;
} else {
    redirect_host = host;
}

var mobile_browser = is_mobile_browser(r.variables.http_user_agent);
var desktop_browser = !mobile_browser;
var cookie_stopMobileRedirect = r.variables.cookie_stopMobileRedirect;
var cookie_stopDesktopRedirect = r.variables.cookie_stopDesktopRedirect;

if (mobile_browser && arg_mobileaction !== 'toggle_view_desktop' && cookie_stopMobileRedirect !== 'true') {
    redirect_host = mobile_host;
}
if (desktop_browser && arg_mobileaction !== 'toggle_view_mobile' && cookie_stopDesktopRedirect !== 'true') {
    redirect_host = desktop_host;
}

// set $stop_redirect_cookie

if (mobile_browser && arg_mobileaction === 'toggle_view_desktop') {
    r.variables.stop_redirect_cookie = ['stopMobileRedirect=true', 'domain=' + wildcard_host(r), 'path=/', 'Max-Age=86400', 'HttpOnly', ''].join(';')
} else if (desktop_browser && arg_mobileaction === 'toggle_view_mobile') {
    r.variables.stop_redirect_cookie = ['stopDesktopRedirect=true', 'domain=' + wildcard_host(r), 'path=/', 'Max-Age=86400', 'HttpOnly', ''].join(';')
} else {
    r.variables.stop_redirect_cookie = '';
}
// ...

this allow by default redirect users to proper version of site, bit also this allow users to manually switch between desktop and mobile versions in any time.

I suppose what such simple logic can be also implemented using varnish.

Having separate domains complicates analytics and search engine optimisation.

Search engines already know about mobile and desktop versions of wikipedia and understand difference.

See https://developers.google.com/search/mobile-sites/mobile-seo/separate-urls for details.

The m. subdomain does not appear to reflect best current practice. Many websites are able to detect mobile clients and deliver the correct variant through an unmodified URL. For example, WikiTravel does this, using CloudFront and MediaWiki/MobileFrontend.

You are talking about https://wikitravel.org/en/Main_Page site?

Looks like this site is broken. I set mobile user agent, but this site still display me desktop version.

Also I don't see any ability to change from desktop to mobile site or from mobile site to desktop site manually.

I propose removing the m. subdomain. The frontend would detect the desired device variant based on UA and request cookies, and send it to MediaWiki in a single header, and would vary the resulting cache based on this request header. MediaWiki (MobileFrontend, or core after T100402 is fixed) would use this request header to select the skin.

If I use desktop browser - how I can manually switch to mobile version of site, is it possible after implementing your proposal?

Also, If I use mobile browser - how I can manually switch to desktop version of site, is it possible after implementing your proposal?

If answer is "No" - removing m. subdomain is bad idea, from my point of view.

@Koavf, ok I see, it work. But WikiTravel don't allow me to browse mobile version of site if I have desktop browser and WikiTravel don't allow me to browse desktop version of site if I use mobile browser.

It permanently show mobile version for mobile users and desktop version for desktop users, and don't remember user choice. This is bug or feature?

For example, even if I manually switch to mobile version of page - https://wikitravel.org/wiki/en/index.php?title=Western_Sahara&mobileaction=toggle_view_mobile - all links on this page, for example https://wikitravel.org/en/North_Africa show me desktop version.

If I use desktop browser - how I can manually switch to mobile version of site, is it possible after implementing your proposal?

Also, If I use mobile browser - how I can manually switch to desktop version of site, is it possible after implementing your proposal?

Yes, you will still be able to do this, and it will work the same way as today. Nothing is changing about this.

Links shared on social media are randomly mobile or non-mobile, and so desktop users often accidentally visit the mobile site.

can be easily resolved by detecting desktop/mobile client by User-Agent and 302 redirecting by default desktop clients from mobile version of site to desktop version of site. [..]

Serving 50% of our social-traffic (or more) with redirects is not an acceptable resolution from a performance perspective.

Current cache proxy logic
handle (cookies and mode-switch logic) {
  
}

if (domain is normal) {
  if (isMobileUserAgent) {
    exit (redirect to mobile domain);
    # user gets redirected...
    # browser makes new URL request over network ...
    # browser URL request arrives in our data centre, and we start another instance of this cache proxy logic ...
  }
}

if (domain is mobile) {
  exit (route to MediaWiki with MobileFrontend=on);
} else {
  exit (route to MediaWiki with MobileFrontend=off);
}
Proposed cache proxy logic
handle (cookies and mode-switch logic) {
  
}

if (isMobileUserAgent) {
  exit (route to MediaWiki with MobileFrontend=on);
} else {
  exit (route to MediaWiki with MobileFrontend=off);
}

See also T214998#5038999. Basically:

  • Better performance for our mobile users, by no longer forcing redirects for many of their page views.
  • Better UX for our desktop users, by no longer serving the mobile site when they do not expect this (random switches based on sharer's url).
  • Nothing else changes for users.

If I use desktop browser - how I can manually switch to mobile version of site, is it possible after implementing your proposal? Also, If I use mobile browser - how I can manually switch to desktop version of site, is it possible after implementing your proposal?

Yes, you will still be able to do this, and it will work the same way as today. Nothing is changing about this.

This proposal already implemented on WikiTravel site, but browsing mobile version of site is not possible using browser with desktop User-Agent, and browsing desktop version of site is not possible using browser with mobile User-Agent. Manually user can switch mobile/desktop view only for each page, but globally, for entire site - can't:

For example, even if I manually switch to mobile version of page - https://wikitravel.org/wiki/en/index.php?title=Western_Sahara&mobileaction=toggle_view_mobile - all links on this page, for example https://wikitravel.org/en/North_Africa show me desktop version.

If this is bug in WikiTravel configuration - probably this bug should be fixed before removing m. subdomain from all Wikipedia sites?

See also T214998#5038999.

As I understand, T214998#5038999 proposal forces mobile users to browse only mobile version of site and forces desktop users to browse only desktop version of site without ability to select which version of site user want to see and browse.

If mobile and desktop versions of site has the same uri - remembering user choice is only possible via cookies, something like this:

userPreferredVersion=mobile or userPreferredVersion=desktop and logic should be something like this:

Proposed cache proxy logic - version 2.0
# user choice has higher priority
if (cookie userPreferredVersion exists) {
  if (cookie userPreferredVersion === 'mobile') {
    exit (route to MediaWiki with MobileFrontend=on);
  }
  if (cookie userPreferredVersion === 'desktop') {
    exit (route to MediaWiki with MobileFrontend=off);
  }
}
# default fallback, if no userPreferredVersion cookie exists
# or if it value is not 'desktop' or 'mobile'
if (isMobileUserAgent) {
  exit (route to MediaWiki with MobileFrontend=on);
} else {
  exit (route to MediaWiki with MobileFrontend=off);
}

[..] it will work the same way as today. Nothing is changing about this.

This proposal already implemented on WikiTravel site, but [..]. If this is bug in WikiTravel configuration - probably this bug should be fixed before removing m. subdomain from all Wikipedia sites?

WikiTravel have done something similar, but their architecture is quite different. It is not comparable. We do not have this bug currently on Wikipedia, and we will not introduce this bug as part of this change. You can rest assured it will keep working. The code for it already exists, it works fine today, and will continue to work. It will not change with this proposal.

Regarding your code suggestion, note that this Phabricator task is not a code review. We are not yet at the implementation stage. I simplified the code in my comment so that it can explain the impact to a wider audience. The actual Wikipedia configuration is in a different syntax and more complex. If and when the plan is accepted, the team implementing it will apply the changes as necessary. Then, you can review the code in the Gerrit.

Krinkle updated the task description. (Show Details)
Krinkle removed a subscriber: wikibugs-l-list.

What does this mean for me as for a user who edits Wikimedia projects from mobile using desktop view for almost 9 years? Does it mean I would not be able to continue doing so and would have to stick to limited in functionality mobile view instead? (Well, in Firefox for Android there is also a "require desktop view" flag, but I am not sure if I can use this preference per web-site and thus it is an undesirable way).

It could be a good idea as long as there is a preference to opt out of mobile view for good for registered users and a long living cookie remembering the view used for anonymous users. Otherwise it sounds like a terrible idea.

What does this mean for me as for a user who edits Wikimedia projects from mobile using desktop view for almost 9 years?

The proposal here is to provide the current mobile experience by default to visitors with detected mobile devices, and the current desktop experience to visitors with detected desktop devices, with the ability to switch experiences back and forth. Logged in users would continue to get the skin of their preference. However, it's too early to make commitments about exactly how this would work and what the experience would be, if we did this.

@Jdlrobson If I understand correctly, you're saying that it is considered a requirement that a user can still find an easy way into the mobile/minerva experience from a desktop browser and stay there when browsing around. In other words, what they can do today by manually going to en.m.wikipedia.org, which they presumably have stumbled upon by accident or figured out by reproducing the url they see on mobile.

Preserving this seems fair indeed. However, to what extent is this not already satisfied by the skin preference and/or the "Mobile view" footer link? The latter also works with a session-bound cookie for logged-out users. And the former is a sticky preference that works across devices for logged-in users.

This change would affect MobileFrontend and therefore make you an important stakeholder. However in terms of resourcing, this seems like it would mainly involve CDN and site configuration changes which CPT, SRE and Perf could handle I think. Are there changes required within MobileFrontend to make this change possible? If this is indeed not a priority for Readers Web currently, it'd be good to know roughly what work is expected to be needed in MobileFrontend.

Krinkle renamed this task from Remove .m. subdomain, serve mobile and desktop variants through the same URL to RFC: Remove .m. subdomain, serve mobile and desktop variants through the same URL.Apr 4 2020, 2:34 AM

Have been somewhat expecting this to be fixed for the better part of 7+ years now. Hopefully this can be implemented soon!

The swap of Traffic for Traffic-Icebox in this ticket's set of tags was based on a bulk action for all tickets that aren't are neither part of our current planned work nor clearly a recent, higher-priority emergent issue. This is simply one step in a larger task cleanup effort. Further triage of these tickets (and especially, organizing future potential project ideas from them into a new medium) will occur afterwards! For more detail, have a look at the extended explanation on the main page of Traffic-Icebox . Thank you!

Note for those interested that this went live for Wikifunctions two weeks ago, per T349408. There'd be more to do for wikitext-based wikis, so this isn't "do it everywhere now", but it's a marker.