Page MenuHomePhabricator

Google search traffic sending users to the wrong Chinese language versions of Wikipedia
Closed, ResolvedPublic

Description

Request Status: New Request
Request Type: break-fix
Related OKRs:

Request Title: Google search traffic sending users to the wrong Chinese language versions of Wikipedia

  • Request Description:

Our counterparts at Google reached out to say their engineering team is having a technical issue where Google Search results are displaying or linking to the wrong versions of Chinese Wikipedia.

Specifically, the technical issues cited by the Google team are:
When Chinese Wikipedia makes on-the-fly C2C conversion (plus some word replacement), the Google system can't tell which Chinese variant it is receiving unless href="/wiki/....." is replaced with href="/zh-{region}/...." It seems this isn't working because of the automatic referrer-dependent redirection.

Google has asked if it would be possible for Wikipedia to stop redirecting zh.wikipedia.org/zh-{region,script}/<subject> coming from Google search (or Google crawler) to zh.wikipedia.org/wiki/<subject> because as long as this redirects Google cannot send its users to the right version of Chinese Wikipedia.

We can also arrange a meeting to troubleshoot this together with their team if that would help expedite a solution.

Let me know if there's any information missing and I'd be happy to share it or find it. Thanks!

  • Indicate Priority Level: High
  • Main Requestors: @Nicholas_Perry
  • Ideal Delivery Date: As soon as possible
  • Stakeholders: Internal: Not sure External: Chinese language users of Google Search

Request Documentation

Document TypeRequired?Document/Link
Related PHAB TicketsYesT54429: Canonical URL should include language variant T108443#6279559 T294716: hreflang Tags' alternate URLs are not fully-qualified
Product One PagerYes<add link here>
Product Requirements Document (PRD)Yes<add link here>
Product RoadmapNo<add link here>
Product Planning/Business CaseNo<add link here>
Product BriefNo<add link here>
Other LinksNo<add links here>

Event Timeline

Nicholas_Perry created this task.

@Nicholas_Perry: Please semantically connect task dependencies via subtasks/parent tasks. Thanks! :)

I think the problem is caused by https://zh.wikipedia.org/wiki/MediaWiki:Gadget-variant-link-fix.js

Google has asked if it would be possible for Wikipedia to stop redirecting zh.wikipedia.org/zh-{region,script}/<subject> coming from Google search (or Google crawler) to zh.wikipedia.org/wiki/<subject>

I have disable it for logged-out users https://zh.wikipedia.org/w/index.php?diff=71419787. Hope this helps.

@Nicholas_Perry About the "because as long as this redirects Google cannot send its users to the right version of Chinese Wikipedia.", the flight redirect (MediaWiki:Gadget-variant-link-fix.js) was created precisely because Google can't take users to the correct version. At least in the past, Google frequently offered the hard-coded variants in result URLs that users don't unfamiliar / non-preferred variant, and users complained that they saw incorrect results. Chinese Wikipedia wants every user to come to '/wiki/' pages and apply variant based on user preferences, not a mandatory version or even a mixed variants (/zh/) that Google thinks the user want.

As well, Google recently began sending desktop users to the mobile site of Wikipedia in the results, which is unfamiliar and fewer features, some users have to try to switch the url manually, which should be discussed together.

The ideal way should be taking all users to '/wiki/' pages, and not only apply variant based on user preferences, but also apply mobile version of the site on mobile device if possible.

IMHO, different chinese variant pages shouldn't be treated as the same page. Right now we set canonical url to /wiki/ page for all chinese variants pages. This makes Google think that they are the same page so that Google only shows one kind of search result. But the reality is different variants should have different search result (title, description). A simplified chinese user should be able to see simplified page title in search results. A traditional chinese user should see the traditional ones.

I propose that we remove canonical url in chinese variant pages (/zh, /zh-tw, /zh-cn). Let Google search do their work to show the right variant search results. When a user click on a /zh-tw search result link, they are taken to the /zh-tw page directly. And if a user visits /wiki/ page directly, we still show the preferred variant based on user preferences.

Change 879579 had a related patch set uploaded (by Winston Sung; author: Winston Sung):

[mediawiki/core@master] Make meta canonical URL language-variant-aware

https://gerrit.wikimedia.org/r/879579

在T305540#7893964中,@fireattack写道:

Lately I noticed Google even start to show mobile link for zh.wiki.

https://www.google.com/search?q=%E6%9C%BA%E7%94%B2%E5%B0%8F%E5%AE%9D+Wikipedia

The first search result is https://zh.m.wikipedia.org/zh-hans/%E9%93%81%E7%94%B2%E5%B0%8F%E5%AE%9D despite I use desktop browser.

related task: T307359

Jdlrobson claimed this task.
Jdlrobson added subscribers: NatHillard, Jdlrobson.

We (@Nicholas_Perry @NatHillard ) talked to a google representative about this on Wednesday 22nd and this is resolved to their satisfaction. Nicholas please let me know if that's not the case.

Should be resolved with or without the linked patch? It got +2 but the parent change was still pending review.

The main crux of this issue has been solved with T294716
https://gerrit.wikimedia.org/r/c/mediawiki/core/+/879579 covers various related edge cases.

I'm still working on making sure this gets merged outside this ticket, but I'm happy to reopen this ticket until that's done if preferred.

Change 879579 merged by jenkins-bot:

[mediawiki/core@master] OutputPage: Fix the behavior for canonical URL and alternate URLs

https://gerrit.wikimedia.org/r/879579