Page MenuHomePhabricator
Create Task
Maniphest T28115

Interwiki redirects via URL and other 301s are indexed by Google search
Open, MediumPublic
Actions

Description

Searching for Wikimedia Commons on Google.nl ( http://www.google.com/search?client=safari&rls=en&q=wikimedia+commons / language: dutch ) the top result is this:

  • Wikimedia Commons

The IUCN Red List has agreed to license distribution maps derived from the data present on their website with a Creative Commons By-SA 2.0. ...
nl.wikipedia.org/wiki/Commons: - Cached

That's an interwiki link, probably indexed by either a double interwiki (ie. en.wikipedia linking to nl:commons: )

Perhaps make these kind of urls, for interwikis that are not limited to local (such as bugzilla: and mw:), a 301 redirect instead ?

Version: 1.21.x
Severity: normal
See Also: T48424, T33838, T38174, T67402.

Details

Reference
bz26115

Related Objects

Event Timeline

• bzimport raised the priority of this task from to Medium.Nov 21 2014, 11:20 PM
• bzimport added a project: MediaWiki-Redirects.
• bzimport set Reference to bz26115.
• bzimport added a subscriber: Unknown Object (MLST).
Krinkle created this task.Nov 25 2010, 9:52 AM
MarkAHershberger added a comment.Feb 12 2011, 7:10 AM

works for me now

Krinkle added a comment.Jan 14 2013, 5:55 PM

Re-opening.

https://www.google.com/search?q=proofread+extension

  1. Extension:Proofread Page - MediaWiki

    url: en.wikipedia.org/wiki/Mw:Extension:Proofread_Page

    The Proofread Page extension can render a book either as a column of OCR text beside a column of scanned images, or broken into its logical organization ...
Krinkle added a comment.Jan 14 2013, 5:57 PM
  • Bug 28242 has been marked as a duplicate of this bug. ***
Krinkle added a comment.Jan 14 2013, 6:00 PM

Clarifying subject.

Bug is previously known as:
" nl.wikipedia.org/wiki/commons:Main_Page indexed by Google "

Eloquence added a comment.Jan 20 2013, 5:41 AM

As of r84820 MediaWiki has been sending 301s, so changing bug title accordingly.

$ wget -S http://en.wikipedia.org/wiki/Mw:Extension:Proofread_Page

(...)
HTTP/1.0 301 Moved Permanently

Tim, any idea why this bug might be (re-)appearing?

Krinkle added a comment.Feb 3 2013, 4:11 AM

Apparently not just for interwiki redirects

q=wikispecies

  1. Wikispecies, free species directory

    url: species.wikipedia.org/Wikimedia

    Foundation project that aims to catalog all species.

Is Google just being stupid and treating all 301s are 302s now?

Nemo_bis added a comment.Feb 11 2013, 6:43 AM

This is not only about interwikis: I just found
MediaWiki-commits Info Page
mail.wikipedia.org/mailman/.../mediawiki-cvs

$ curl -i http://mail.wikipedia.org/mailman/listinfo/mediawiki-cvs
HTTP/1.1 301 Moved Permanently
Location: https://lists.wikimedia.org/mailman/listinfo/mediawiki-cvs

(In reply to comment #6)

Is Google just being stupid and treating all 301s are 302s now?

Yes?

Eloquence added a comment.Feb 12 2013, 2:12 AM

I've sent an inquiry to our contacts at Google about this.

Eloquence added a comment.Feb 19 2013, 5:41 AM

They're looking into it as of a few days ago; haven't heard anything more specific yet.

Nemo_bis added a comment.Feb 20 2013, 1:57 PM

Is canonical url a related issue?
For instance:
<link rel="canonical" href="//commons.wikimedia.org/wiki/File:Harshad_GNUnify_2013.jpg" />
But first result on Google for "Harshad GNUnify" is the file description on en.wikipedia.org.

Eloquence added a comment.Mar 2 2013, 8:48 AM

All examples in the bug now return the correct URLs for me. Resolving as fixed for now.

He7d3r added a comment.Mar 2 2013, 6:25 PM

(In reply to comment #11)

All examples in the bug now return the correct URLs for me. Resolving as
fixed
for now.

The search
https://www.google.com/search?q=proofread+extension
still returns
http://en.wikipedia.org/wiki/Oldwikisource:Wikisource:ProofreadPage
which redirects to
http://wikisource.org/wiki/Wikisource:ProofreadPage

There are lots of similar examples here:
https://www.google.com/search?q=intitle%3AWikibooks+inurl%3Awikipedia.org%2Fwiki%2Fb%3A

Aklapper added a comment.Mar 5 2013, 10:45 AM

Entering

proofread extension inurl:oldwikisource

in Google, the only results I get are

  • en.wikipedia.org/wiki/Oldwikisource:Wikisource:Scriptorium
  • en.wikipedia.org/wiki/Oldwikisource:Wikisource:2007
Aklapper added a comment.Mar 21 2013, 3:22 PM

Helder: Do you still see the problem in comment 12?
As I wrote in comment 13, I cannot reproduce.

He7d3r added a comment.Mar 23 2013, 6:06 PM

The first result when I search for "Cookbook:Herbal Iced Tea" is
http://en.wikipedia.org/wiki/Wikibooks:Cookbook:Herbal_Iced_Tea
which redirects to
http://en.wikibooks.org/wiki/Cookbook:Herbal_Iced_Tea

Aklapper added a comment.Mar 25 2013, 8:43 AM

I can confirm comment 15 on google.com.

Nemo_bis added a comment.May 29 2013, 3:44 PM

I checked this bug a few weeks ago and it was still current; now Erik says, "Actually none of the examples in the bug still return the reported results for me" (thanks).
http://thread.gmane.org/gmane.org.wikimedia.foundation/66428/focus=66464
Indeed they seem all resolved now, with a single probably irrelevant exception below. But why this oscillating? :/

(In reply to comment #10)

But first result on Google for "Harshad GNUnify" is the file description on
en.wikipedia.org.

That's fixed; second is now en.zero.wikipedia.org/.../File:Harshad_GNUnify , different bug.

(In reply to comment #12)

There are lots of similar examples here:
https://www.google.com/search?q=intitle%3AWikibooks+inurl%3Awikipedia.
org%2Fwiki%2Fb%3A

Now I only see fr.wikipedia.org/wiki/b:Utilisateur:Airridi‎ here

• Mattflaschen-WMF added a comment.May 29 2013, 6:29 PM

I don't see the Airridi‎ example on the last one either. All of the results are about copying to Wikibooks, etc.

Reopen if you have an example that still occurs.

He7d3r added a comment.May 29 2013, 8:41 PM

The query

inurl:"commons:category" site:en.wikipedia.org

returns a few examples:

https://en.wikipedia.org/wiki/commons:Category:Maps_of_Ireland‎
https://commons.wikimedia.org/wiki/Category:Maps_of_Ireland

https://en.wikipedia.org/wiki/Commons:Category:Range_maps‎
https://commons.wikimedia.org/wiki/Category:Range_maps

http://en.wikipedia.org/wiki/Commons:Category:Chicken‎
http://commons.wikimedia.org/wiki/Category:Chicken

• Mattflaschen-WMF added a comment.Jun 10 2013, 10:14 PM

There's still some at https://www.google.com/search?q=inurl:%22wiki/commons:%22%20site:en.wikipedia.org , so this is not fully fixed.

He7d3r added a comment.May 7 2014, 7:33 PM

https://www.google.com.br/search?q="codeEditor.configure"+site%3Awikipedia.org
shows
en.wikipedia.org/wiki/mw:Extension:CodeEditor‎
as its first result

Nemo_bis added a comment.Nov 6 2014, 7:41 AM

Still happening. https://archive.today/ysbIZ

• Mattflaschen-WMF added a comment.Nov 13 2014, 11:10 PM

It may or may not still be happening. But I don't see any examples at https://archive.today/ysbIZ .

Nemo_bis added a comment.Nov 14 2014, 12:12 AM

(In reply to Matthew Flaschen from comment #23)

It may or may not still be happening. But I don't see any examples at
https://archive.today/ysbIZ .

Don't you see a list of meta.wikipedia.org URLs? Perhaps you didn't notice the "p"?

Krinkle added a comment.Jan 15 2015, 10:28 PM

As of January 2015 this is still happening.

https://www.google.co.uk/search?q=tablesorter+site%3Amediawiki.org

Result #3: Help:Sorting - Meta - Meta-Wiki - Wikimedia
https://www.mediawiki.org/wiki/meta:Help:Sorting
The JavaScript code jquery.tablesorter.js (source) of the tablesorter is loaded by the ResourceLoader. Some sites may have a page MediaWiki:Common.js ...

Nemo_bis mentioned this in T87561: Google indexes non-canonical m.wikipedia.org URLs.Jan 26 2015, 11:52 AM
jayvdb subscribed.Jan 26 2015, 9:10 PM
jayvdb mentioned this in T75854: Fix provided search results in Wikimedia Phabricator.Feb 15 2015, 4:49 AM
jayvdb added a comment.Feb 15 2015, 4:52 AM

A slight variation, I am seeing :en: links in Google results.

https://www.google.com.au/search?q=Elections+in+South+America+site:commons.wikimedia.org

results:

commons.wikimedia.org/wiki/:en:Argentina
commons.wikimedia.org/wiki/:en:José%20Luis%20Rodríguez%20Zapatero
commons.wikimedia.org/wiki/:en:Ohio
commons.wikimedia.org/wiki/:en:North%20Carolina
commons.wikimedia.org/wiki/:en:KwaZulu-Natal
commons.wikimedia.org/wiki/:en:Gibraltar
etc

Krinkle added a comment.EditedMar 3 2015, 4:48 AM

Even for en.wikipedia.org, pointing to itself via en: and w:en:.

https://www.google.co.uk/search?q=special+export+delhi+wikipedia

w:en:Special:Export/Delhi - Wikipedia
https://en.wikipedia.org/wiki/en:Special:Export/Delhi
Wikipedia enwiki https://en.wikipedia.org/wiki/Main_Page MediaWiki first-letter Media Special Talk User User talk Wikipedia Wikipedia talk File File ...

See T91363 for the issue with Special:Export in general (even when not behind interwiki redirect).

Krinkle merged a task: T30242: Pages should not be indexed by search engines through interwiki links from other wikis.Mar 3 2015, 4:56 AM
Krinkle added a subscriber: Bawolff.
MarkAHershberger unsubscribed.Mar 3 2015, 3:31 PM
Nemo_bis updated the task description. (Show Details)Jun 17 2015, 9:41 AM
Nemo_bis added a project: Traffic.
Nemo_bis set Security to None.
Nemo_bis mentioned this in T67402: URLs for the same title without extra query parameters should have the same canonical link.Jun 23 2015, 8:34 AM
Nemo_bis edited projects, added SEO; removed Traffic.Jun 23 2015, 8:53 AM
Florian subscribed.Jun 23 2015, 10:56 AM
jeremyb-phone added a subscriber: jeremyb.Jun 23 2015, 3:05 PM
Krinkle moved this task from General to Response on the MediaWiki-Redirects board.Jul 31 2017, 9:29 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL