Page MenuHomePhabricator

Wiki pages with a noindex meta tag are indexed through rest.wikimedia.org
Closed, DuplicatePublic

Description

For instance, https://fr.wikipedia.org/wiki/Utilisateur:Exemple has a tag that prevents indexing, because the user namespace is not indexed on frwiki. However, http://rest.wikimedia.org/fr.wikipedia.org/v1/page/html/Utilisateur:Exemple does not have such a tag. Some other pages in this namespace are actually indexed by Google.

There should be either a <meta> tag or a X-Robots-Tag header to prevent that.

Event Timeline

Orlodrim raised the priority of this task from to Needs Triage.
Orlodrim updated the task description. (Show Details)
Orlodrim added a project: RESTBase.
Orlodrim subscribed.