Page MenuHomePhabricator

robots.txt
Closed, ResolvedPublic

Description

beta robots.txt has been made to disallow anything (bug 37189). The Apaches are currently serving the file from /home/wikipedia/common/robots.txt which is tracked in git.

We should make the Apache conf to alias robots.txt to another, labs only, file.


Version: unspecified
Severity: normal

Details

Reference
bz38217

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 12:58 AM
bzimport set Reference to bz38217.
bzimport added a subscriber: Unknown Object (MLST).

Was merged a few time ago already. Note that each wiki can use [[Mediawiki:robots.txt]] to override the default generated robots.txt.