The content of [[commons:MediaWiki:Robots.txt]] does not seem to show up at http://commons.wikimedia.org/robots.txt . It should, shouldn't it? Is something wrong with the configuration?
Version: unspecified
Severity: normal
URL: http://commons.wikimedia.org/robots.txt
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Invalid | Feature | None | T16720 robots.txt (tracking) | ||
| Resolved | None | T24040 MediaWiki:Robots.txt has no effect on Commons |
It seems [[meta:MediaWiki:Robots.txt]] isn't either. Perhaps because they have a pedia suffix?
Yeah, looks like the relevant RewriteRules are only present in http://noc.wikimedia.org/conf/main.conf but not in http://noc.wikimedia.org/conf/remnant.conf where Commons and Meta are. Sprinkling a few suitably placed copies of
into that file would seem likely to fix it. (Also should probably do the same for the other relevant *.conf files.)
CCing JeLuF on this -- he fixed bug 15663, so I assume he knows something about this robots.txt setup. ;)
Those files have interesting data, like the non-working wikipedia.geo.blitzed.org ServerAlias, or the robots file name. Why isn't robots.php in svn?