Maniphest T193158

Detect unsuitable archive snapshots
Open, LowPublic
Actions

Assigned To

None

Authored By

	Cirdan
	Apr 26 2018, 12:09 PM

Description

While it is not (yet) possible for the IABot to check whether an archive snapshot is a suitable source, there are cases where IABot adds a link to a snapshot which is broken in a way which should be reasonably easy to detect.

This task collects these cases and gives an overview.

Related Objects
Search...

Status	Assigned	Task
Open	None	T193158 Detect unsuitable archive snapshots
Invalid	Cyberpower678	T193129 IABot inserts links to archive snapshots with broken document viewer
Resolved	Cyberpower678	T191277 InternetArchiveBot should detect blank snapshots
Resolved	Cyberpower678	T194604 InternetArchiveBot should not link to archive snapshots which have been excluded from the Wayback Machine
Declined	None	T245902 InternetArchiveBot should detect and not link to snapshots of domain reselling/domain squatting pages

Event Timeline

Cirdan created this task.Apr 26 2018, 12:09 PM

Restricted Application assigned this task to Cyberpower678. · View Herald TranscriptApr 26 2018, 12:09 PM

Cirdan renamed this task from Automatically detect unsuitable archive snapshots to Detect unsuitable archive snapshots.Apr 26 2018, 12:09 PM

Cirdan added a subtask: T193129: IABot inserts links to archive snapshots with broken document viewer.

Cirdan added a subtask: T191276: InternetArchiveBot adds link to archived version which is not displayed due to robots.txt.

Cirdan added a subtask: T191277: InternetArchiveBot should detect blank snapshots.

Cirdan moved this task from Inbox to New feature on the InternetArchiveBot board.Apr 26 2018, 12:11 PM

Cirdan removed Cyberpower678 as the assignee of this task.Apr 26 2018, 5:50 PM

Cirdan updated the task description. (Show Details)

Cirdan added a subscriber: Cyberpower678.

Cirdan updated the task description. (Show Details)May 13 2018, 6:41 PM

I believe that these issues can also be solved by running a third-party service which has the necessary access through IABot's API. At least for T191276 and T194604, this should be straightforward and I'm happy to try creating a small script once 2.0 is released.

Cirdan triaged this task as Low priority.May 17 2018, 2:28 PM

Cirdan mentioned this in T191277: InternetArchiveBot should detect blank snapshots.Jun 2 2018, 8:07 AM

• Vvjjkkii renamed this task from Detect unsuitable archive snapshots to t5daaaaaaa.Jul 1 2018, 1:13 AM

• Vvjjkkii raised the priority of this task from Low to High.

• Vvjjkkii added projects: CheckUser, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), Tamil-Sites, Gamepress, Hashtags, Jade, KartoEditor, Language-2018-Apr-June, New-Editor-Experiences, Mail, archived--TCB-Team.

• Vvjjkkii updated the task description. (Show Details)

Green_Cardamom renamed this task from t5daaaaaaa to Detect unsuitable archive snapshots.Jul 1 2018, 4:30 AM

Green_Cardamom lowered the priority of this task from High to Low.

Green_Cardamom removed projects: archived--TCB-Team, Mail, New-Editor-Experiences, Language-2018-Apr-June, KartoEditor, Jade, Hashtags, Gamepress, Tamil-Sites, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), CheckUser.

Green_Cardamom updated the task description. (Show Details)

Cirdan added a project: Tracking-Neverending.Jul 18 2018, 5:55 AM

(Removing Tracking-Neverending tag as per its description. Maybe you meant Epic or something similar.)

Cirdan added a project: Epic.Jul 18 2018, 8:23 AM

Cirdan removed a subtask: T191276: InternetArchiveBot adds link to archived version which is not displayed due to robots.txt.Jul 21 2018, 6:11 AM

Aklapper removed a subscriber: Aklapper.Jul 21 2018, 11:02 AM

My bot WaybackMedic does check for these (robots.txt, excluded from the archive, many other cases) and is able to replace them with new archives at a different provider.

Cyberpower678 closed subtask T194604: InternetArchiveBot should not link to archive snapshots which have been excluded from the Wayback Machine as Resolved.Oct 27 2018, 4:40 PM

Cirdan mentioned this in T217687: www.thegridto.com is dead.Mar 6 2019, 6:14 AM

Cyberpower678 closed subtask T193129: IABot inserts links to archive snapshots with broken document viewer as Invalid.Mar 6 2019, 8:49 PM

Thryduulf added a subtask: T245902: InternetArchiveBot should detect and not link to snapshots of domain reselling/domain squatting pages.Feb 22 2020, 3:36 AM

Cyberpower678 closed subtask T245902: InternetArchiveBot should detect and not link to snapshots of domain reselling/domain squatting pages as Declined.Sep 1 2020, 12:56 AM

Harej moved this task from New feature to Big projects on the InternetArchiveBot board.Jan 22 2021, 10:03 PM

Cyberpower678 closed subtask T191277: InternetArchiveBot should detect blank snapshots as Resolved.Feb 1 2021, 12:56 AM