Page MenuHomePhabricator

Entries to robots.txt to block deletion requests at es:
Closed, ResolvedPublic


Similarly to bug 4776, i ask blocking to robots eswiki deletion requests.

Disallow: /wiki/Wikipedia:Consultas_de_borrado/

Version: unspecified
Severity: enhancement



Event Timeline

bzimport raised the priority of this task from to Medium.Nov 21 2014, 9:18 PM
bzimport set Reference to bz6746.
bzimport added a subscriber: Unknown Object (MLST).

manolovarg wrote:

Please add the following for es.wikipedia robots.txt. We have had problems with google page rank that amplifies these
nonencyclopedic pages
of WP and we have had usuary that have complained with us. Thanks

Disallow: /wiki/Usuario_Discusión:*
Disallow: /wiki/Usuario_Discusi%C3%B3n:*
Disallow: /wiki/Usuario:*
Disallow: /wiki/Wikipedia:Caf%C3%A9/Portal/Archivo/
Disallow: /wiki/Wikipedia:Caf%C3%A9/Portal/Archivo/Miscel%C3%A1nea/
Disallow: /wiki/Wikipedia:Café/Portal/Archivo/Miscelánea/

Today it was talked again
Please, Disallow: /wiki/Wikipedia:Consultas_de_borrado/

The other pages Manolo comments are probably not neccesary to block for now (and
in case of needing to block them, would be incomplete).

Some rules above have wildcards. The robots exclusion protocol doesn't permit wildcards or regexes or anything like that. * for a user agent entry is special cased. It just matches against the beginning of the path desired without paying attention to anything that follows. So, for example from your request:

Disallow: /wiki/Wikipedia:Caf%C3%A9/Portal/Archivo/
Disallow: /wiki/Wikipedia:Caf%C3%A9/Portal/Archivo/Miscel%C3%A1nea/

The second entry is blocked already by the first entry.

See also

You're right. However, i don't see why /wiki/Wikipedia:Caf%C3%A9/Portal/Archivo/ should be blocked (as opposed to /wiki/Wikipedia:Consultas_de_borrado/ with which have been problems)