quarry: Add a robots.txt
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	taavi
	Jun 20 2025, 8:58 AM

Description

Quarry should have a robots.txt to control well-behaving bots. At least the login route should be blocked, but I'm sure there are some more (e.g. forking?).

Event Timeline

taavi created this task.Jun 20 2025, 8:58 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 20 2025, 8:58 AM

The utility of a public web search returning cached SQL query results from Quarry seems pretty low. I suppose an argument could be made for public web search making finding an existing query to fork easier, but the app does have an internal search for that use case. I would be tempted to set a Disallow: / policy.

taavi claimed this task.Jun 23 2025, 10:16 AM

supertassu opened https://github.com/toolforge/quarry/pull/92

supertassu closed https://github.com/toolforge/quarry/pull/92

taavi closed this task as Resolved.Jun 23 2025, 10:47 AM

Mentioned in SAL (#wikimedia-cloud) [2025-06-23T10:49:20Z] <taavi> deploy deny-all robots.txt file T397502

quarry: Add a robots.txtClosed, ResolvedPublicActions

Description

Event Timeline

quarry: Add a robots.txt
Closed, ResolvedPublic
Actions