Migrate dead external links to archives
Open, NormalPublic

Tokens
"Burninate" token, awarded by chstdu."Like" token, awarded by Liuxinyu970226."Love" token, awarded by waldyrious."Like" token, awarded by Shizhao."Like" token, awarded by fbstj."Like" token, awarded by Cyberpower678.
Assigned To
Authored By
DannyH, Dec 4 2015

Description

This card tracks a top 10 wish from the Community Wishlist Survey: https://meta.wikimedia.org/wiki/2015_Community_Wishlist_Survey

Original proposal: Most external links have an average lifespan of about 7 years before they go dead. As Wikipedia ages, the dead external links problem grows exponentially. Internet Archive has partnered with Wikipedia to ensure all new external links have a Wayback cache. However there has been no formal process of adding the Wayback links to Wikipedia (via the cite web |archiveurl=.. feature for example). There have been attempts to automate with various bots (see en:WP:Link rot) but the coding is non-trivial and multiple volunteer efforts have stalled. Likely what will be required is a team of programmers working full-time, something that is beyond the scope of a few volunteers working spare time. It's the sort of coding work that MediaWiki could sponsor and make a big difference in the quality of content, impacting every article. -- Green Cardamom (talk) 19:27, 7 November 2015 (UTC)

Community Tech preliminary assessment

Support: High. Dead reference links hurt our projects' reliability and verifiability, and connecting deadlinks with an archive supports the usefulness of our content. There were some dissents in the voting phase, pointing out that it's better when humans find the appropriate alternative links, rather than a bot that might not choose the right one.

Impact: High. Improving the quality of citations helps readers as well as contributors. There are some bots currently running on English, French and Spanish Wikipedias. We want to help build solutions that can be adapted to every language.

Feasibility: High. Cyberbot II is currently active on English Wikipedia, and Elvisor on Spanish Wikipedia. Cyberpower678's work on Cyberbot is being supported by The Wikipedia Library and the Internet Archive. There is obviously good work being done here, and we can figure out how to best support it, and help it to scale globally.

Risk: Low. Cyberbot II is running on English Wikipedia, with no major issues encountered. It may be challenging to integrate with other wikis’ citation templates.

Project page: https://meta.wikimedia.org/wiki/Community_Tech/Migrate_dead_external_links_to_archives

InternetArchiveBot code: https://github.com/cyberpower678/Cyberbot_II

Related Objects

StatusAssignedTask
OpenCyberpower678
ResolvedNone
ResolvedSadads
ResolvedNone
Resolvedkaldari
ResolvedNiharika
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedNiharika
ResolvedNiharika
ResolvedNiharika
ResolvedNiharika
ResolvedNiharika
ResolvedNiharika
ResolvedNiharika
ResolvedNiharika
ResolvedCyberpower678
ResolvedNiharika
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedNiharika
ResolvedNiharika
ResolvedCyberpower678
ResolvedNiharika
ResolvedNiharika
ResolvedCyberpower678
InvalidNiharika
ResolvedNiharika
ResolvedMusikAnimal
ResolvedCyberpower678
OpenCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
OpenCyberpower678
OpenCyberpower678
OpenCyberpower678
OpenCyberpower678
OpenCyberpower678
OpenCyberpower678
OpenCyberpower678
ResolvedCyberpower678
DeclinedNone
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedMusikAnimal
Resolvedbd808
ResolvedMusikAnimal
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
DeclinedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
InvalidNone
ResolvedMusikAnimal
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedCyberpower678
ResolvedNiharika
ResolvedNiharika
OpenCyberpower678
There are a very large number of changes, so older changes are hidden. Show Older Changes
DannyH changed the title from "Migrate dead links to Wayback Machine" to "Migrate dead external links to archives".Feb 11 2016, 12:19 AM
Thoken added a subscriber: Thoken.Mar 3 2016, 4:32 PM
Bedoron added a subscriber: Bedoron.Mar 5 2016, 2:01 PM
Cyberpower678 changed the status of subtask T112881: Create Cyberbot Project on Labs from "Stalled" to "Open".Mar 27 2016, 5:19 AM

Wow, I can't help but reminisce at how crude and ugly the code was a year ago. Now, IABot, is really close to being completed for the English Wikipedia. The bot has definitely come very far in such a short time.

Glaisher removed a subscriber: Glaisher.Jul 25 2016, 4:17 AM
kaldari edited the task description. (Show Details)Aug 1 2016, 8:25 PM
kaldari edited the task description. (Show Details)
Cyberpower678 changed the task status from "Open" to "Stalled".Aug 20 2016, 12:04 PM
Cyberpower678 changed the task status from "Stalled" to "Open".Sep 4 2016, 12:31 AM
Quiddity edited the task description. (Show Details)Nov 16 2016, 8:17 PM
DannyH closed this task as "Resolved".Dec 15 2016, 7:44 PM

Closing this ticket, because Community Tech has finished working on this project. Well done, everyone! :)

DannyH moved this task from Archive to Freezer on the Community-Tech board.Dec 15 2016, 7:52 PM

Okay, sorry about that -- yes, this is still open for your development.

There's a new ticket for Automatic links: T153354: Automatic archive for new external links

chstdu added a subscriber: chstdu.Wed, Mar 22, 3:20 PM