tags in page source
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	mpopov
	Feb 7 2020, 10:19 PM

Description

We (Kate and Mikhail) were investigating canonical vs alternate links for T234340 based on information in https://webmasters.googleblog.com/2019/02/consolidating-your-website-traffic-on.html and were really confused when we were seeing clicks and impressions reported for en.m.wikipedia.org, since

In the current version, some of your traffic is attributed to the canonical property and some to the alternate property. The new version should attribute all of your traffic to the canonical property.

So based on our system and that blog post, we should be seeing 0 and 0 for .m. domains in the Google Search Console (GSC) but that's not the case. Looking at the page source revealed something incredibly weird:

From view-source:https://en.wikipedia.org/wiki/Wikipedia:

<link rel="alternate" href="android-app://org.wikipedia/http/en.m.wikipedia.org/wiki/Wikipedia"/>
<link rel="alternate" type="application/x-wiki" title="Edit this page" href="/w/index.php?title=Wikipedia&amp;action=edit"/>
<link rel="edit" title="Edit this page" href="/w/index.php?title=Wikipedia&amp;action=edit"/>
<link rel="apple-touch-icon" href="/static/apple-touch/wikipedia.png"/>
<link rel="shortcut icon" href="/static/favicon/wikipedia.ico"/>
<link rel="search" type="application/opensearchdescription+xml" href="/w/opensearch_desc.php" title="Wikipedia (en)"/>
<link rel="EditURI" type="application/rsd+xml" href="//en.wikipedia.org/w/api.php?action=rsd"/>
<link rel="license" href="//creativecommons.org/licenses/by-sa/3.0/"/>
<link rel="canonical" href="https://en.wikipedia.org/wiki/Wikipedia"/>
<link rel="dns-prefetch" href="//meta.wikimedia.org" />

and from view-source:https://en.m.wikipedia.org/wiki/Wikipedia:

<link rel="alternate" href="android-app://org.wikipedia/http/en.m.wikipedia.org/wiki/Wikipedia"/>
<link rel="manifest" href="/w/api.php?action=webapp-manifest"/>
<link rel="alternate" type="application/x-wiki" title="Edit this page" href="/w/index.php?title=Wikipedia&amp;action=edit"/>
<link rel="edit" title="Edit this page" href="/w/index.php?title=Wikipedia&amp;action=edit"/>
<link rel="apple-touch-icon" href="/static/apple-touch/wikipedia.png"/>
<link rel="shortcut icon" href="/static/favicon/wikipedia.ico"/>
<link rel="search" type="application/opensearchdescription+xml" href="/w/opensearch_desc.php" title="Wikipedia (en)"/>
<link rel="EditURI" type="application/rsd+xml" href="//en.wikipedia.org/w/api.php?action=rsd"/>
<link rel="license" href="//creativecommons.org/licenses/by-sa/3.0/"/>
<link rel="canonical" href="https://en.wikipedia.org/wiki/Wikipedia"/>
<link rel="dns-prefetch" href="//meta.wikimedia.org" />

For some reason we're using href="android-app://org.wikipedia/http/en.m.wikipedia.org/wiki/Wikipedia", shouldn't that be href="https://en.m.wikipedia.org/wiki/Wikipedia" for indexing to correctly understand which version of the article is canonical vs alternate?

It is our suspicion that this is why we're not seeing what we expect to see in GSC.

QA

NOTE: The patch will be deployed on Thursday, 20th February

Navigate to any page in a content namespace (e.g. mainspace)
Observe that there's no alternate link whos href attribute begins with android-app://org.wikipedia, e.g. android-app://org.wikipedia/http/en.m.wikipedia.org/wiki/Cult_of_Luna

QA Results

AC	Status	Details
1	✅	T244614#5908962
2	✅	T244614#5908962

Details

	Subject	Repo	Branch	Lines +/-
	No longer emit alternate link with android-app scheme.	mediawiki/extensions/MobileFrontend	master	+0 -44

Customize query in gerrit

Related Objects

Mentioned Here: T234807: Determine impact of $wgMFNoindexPages search traffic to arwiki, zhwiki, hiwiki, itwiki, nlwiki, kowiki

Event Timeline

mpopov created this task.Feb 7 2020, 10:19 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 7 2020, 10:19 PM

mpopov updated the task description. (Show Details)Feb 7 2020, 10:35 PM

Jdlrobson added a project: Android-app-Bugs.Feb 9 2020, 11:01 PM

Restricted Application added a project: Wikipedia-Android-App-Backlog. · View Herald TranscriptFeb 9 2020, 11:01 PM

• Charlotte edited projects, added Wikipedia-Android-App-Backlog (Android-app-release-v2.7.30x-O-Ontbijtkoek); removed Wikipedia-Android-App-Backlog.Feb 11 2020, 5:19 PM

@mpopov This url format is in fact correct, for the purpose that it was supposed to serve. But actually this purpose is quite outdated and probably no longer supported on Android. In the old days, this tag basically enabled the following logic:

"if the user has this particular app installed on the device (defined by the package name org.wikipedia) then launch that app explicitly, and don't navigate to this link in the browser."

The thing is that Android has stopped "honoring" this scheme a long time ago, so we can probably remove this alternate link quite safely.

Change 571560 had a related patch set uploaded (by Dbrant; owner: Dbrant):
[mediawiki/extensions/MobileFrontend@master] No longer emit alternate link with android-app scheme.

https://gerrit.wikimedia.org/r/571560

gerritbot added a project: Patch-For-Review.Feb 11 2020, 6:56 PM

Dbrant claimed this task.Feb 11 2020, 8:49 PM

Dbrant moved this task from Android-app-release-v2.7.30x-O-Ontbijtkoek to Tracking on the Wikipedia-Android-App-Backlog board.

Dbrant edited projects, added Wikipedia-Android-App-Backlog; removed Wikipedia-Android-App-Backlog (Android-app-release-v2.7.30x-O-Ontbijtkoek).

Thank you for explaining the motivation and submitting the patch to fix it, @Dbrant!

@Niedzielski @Jdlrobson @ovasileva: Will a separate patch be required to add <link rel="alternate" href="[mobile url]"/> in so that Google correctly indexes mobile pages as alternate to their canonical versions?

phuedx edited projects, added Web-Team-Backlog (Kanbanana-2019-20-Q3); removed Web-Team-Backlog.Feb 14 2020, 1:26 PM

I'm moving this into Needs Code Review after a discussion with @ovasileva.

Change 571560 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] No longer emit alternate link with android-app scheme.

https://gerrit.wikimedia.org/r/571560

For history this was added here: https://github.com/wikimedia/mediawiki-extensions-MobileFrontend/commit/677aa8935ff497674265f2581accc25487d4a977
I didn't see any reason not to remove it at this point.

I think this can skip QA (at least I'm not sure how to QA this other than verify the link is removed from the output HTML when it hits production).

ReleaseTaggerBot added a project: MW-1.35-notes (1.35.0-wmf.20; 2020-02-18).Feb 14 2020, 8:00 PM

Neither config variable removed in https://gerrit.wikimedia.org/r/571560 are used anywhere other than WMF production 👍

MFAppPackageId: https://codesearch.wmflabs.org/search/?q=wgMFAppPackageId
MFAppScheme: https://codesearch.wmflabs.org/search/?q=wgMFAppScheme

In T244614#5885697, @Jdlrobson wrote:

<snip /> at least I'm not sure how to QA this other than verify the link is removed from the output HTML when it hits production

LGTM

phuedx moved this task from Ready for Signoff to Needs QA on the Web-Team-Backlog (Kanbanana-2019-20-Q3) board.Feb 17 2020, 10:08 AM

phuedx reassigned this task from Dbrant to Edtadros.Feb 17 2020, 10:22 AM

phuedx updated the task description. (Show Details)

ovasileva triaged this task as Medium priority.Feb 17 2020, 10:39 AM

In T244614#5874662, @mpopov wrote:

Thank you for explaining the motivation and submitting the patch to fix it, @Dbrant!

@Niedzielski @Jdlrobson @ovasileva: Will a separate patch be required to add <link rel="alternate" href="[mobile url]"/> in so that Google correctly indexes mobile pages as alternate to their canonical versions?

@mpopov - from my understanding this is the same as the test we are doing in T234807: Determine impact of $wgMFNoindexPages search traffic to arwiki, zhwiki, hiwiki, itwiki, nlwiki, kowiki (although please correct me if I'm wrong here) - discussed with @kzimmerman on raising the priority of this analysis - if we don't see any decreases we can go ahead an do this for all wikis.

Test Result

Status: ✅ PASS
OS: macOS Catalina
Browser: Chrome
Device: MBP
Emulated Device: iPhoneX

Test Artifact(s):

QA Steps

Navigate to any page in a content namespace (e.g. mainspace)
Observe that there's no alternate link whos href attribute begins with android-app://org.wikipedia, e.g. android-app://org.wikipedia/http/en.m.wikipedia.org/wiki/Cult_of_Luna

✅ AC1: Desktop

Screen Shot 2020-02-21 at 5.00.38 PM.png (2×1 px, 1 MB)

✅ AC2: Mobile

Screen Shot 2020-02-21 at 5.04.55 PM.png (2×1 px, 1 MB)

Edtadros updated the task description. (Show Details)Feb 22 2020, 1:38 AM

ovasileva closed this task as Resolved.Feb 26 2020, 12:27 PM

	F31626257: Screen Shot 2020-02-21 at 5.00.38 PM.png
	Feb 22 2020, 1:22 AM

	F31626259: Screen Shot 2020-02-21 at 5.04.55 PM.png
	Feb 22 2020, 1:22 AM

Investigate and fix <link rel="alternate"/> tags in page sourceClosed, ResolvedPublicActions