Maniphest T191276

InternetArchiveBot adds link to archived version which is not displayed due to robots.txt
Closed, DeclinedPublic
Actions

Assigned To

Authored By

	Cirdan
	Apr 3 2018, 6:03 AM

Tags

Referenced Files

None

Subscribers

Description

Here IABot added a link to this snapshot which shows a robots.txt warning.

This is either an error, or IABot does not yet check for these cases.

Related Objects

Mentioned In: T193158: Detect unsuitable archive snapshots
Mentioned Here: T16720: robots.txt (tracking)

Event Timeline

Cirdan created this task.Apr 3 2018, 6:03 AM

Restricted Application added a project: Internet-Archive. · View Herald TranscriptApr 3 2018, 6:03 AM

Cirdan added a parent task: T193158: Detect unsuitable archive snapshots.Apr 26 2018, 12:10 PM

Cirdan mentioned this in T193158: Detect unsuitable archive snapshots.Apr 26 2018, 5:51 PM

Cirdan triaged this task as Low priority.May 17 2018, 2:27 PM

Cyberpower678 moved this task from Inbox to Backlog: Other on the InternetArchiveBot board.Jul 18 2018, 3:07 AM

Liuxinyu970226 added a parent task: T16720: robots.txt (tracking).Jul 21 2018, 4:42 AM

T16720 is about MediaWiki/Wikimedia robots.txt configuration issues.

This should be resolved on the Wayback Machine's end now.