Every time InternetArchiveBot encounters a link that is marked as pay-walled, it should add that domain into a dedicated table. It should then check each link that it scans to see if it's domain is listed in the paywall table, and if so skip it.
|Resolved||Cyberpower678||T120433 Migrate dead external links to archives|
|Resolved||Cyberpower678||T132606 Advanced deadlink detection for Cyberbot (tracking)|
|Resolved||Cyberpower678||T132940 Add ability for InternetArchiveBot to skip paywalled links|