Page MenuHomePhabricator

Support anonymous FTP access in CheckIfDead class (Cyberbot)
Closed, ResolvedPublic


Per T136728, sometimes FTP sites require you to log in as a guest in order to access the content (See RFC 1635: How to Use Anonymous FTP). Currently, CheckIfDead::checkDeadlinks() reports these pages as dead even though they aren't. For example,

In order to access these pages, you have to append something like "--user" to the curl request, where the user is "anonymous" and the password is any email address.


$ curl
curl: (67) Access denied: 530
$ curl --user
total 20760
-rw-r--r-- 1 20    1065 Jan 29  2007 README.TXT
-rw-r--r-- 1 20 8361184 Jan  9  2007 Supplementary_Video_1.avi
-rw-r--r-- 1 20 1819942 Jan  9  2007 Supplementary_Video_2.avi
-rw-r--r-- 1 20 3436606 Jan  9  2007 Supplementary_Video_3.avi
-rw-r--r-- 1 20 2239436 Jan  9  2007 Supplementary_Video_4_.avi
-rw-r--r-- 1 20 2037788 Jan  9  2007 Supplementary_Video_5.avi
-rw-r--r-- 1 20 3312198 Jan  9  2007 Supplementary_Video_6.avi

In order to properly support these FTP sites, when CheckIfDead::checkDeadlinks() identifies a URL as an FTP site (which it already does), and the request fails, it should try again with "--user" as part of the curl request. Note that some FTP sites will return an error if you attempt to log in anonymously, so this should only be tried if the first attempt fails.

Please test with the following URLs to make sure they all return false ("alive"):

  • (requires anonymous log in)

And add the first URL above to checkIfDeadTest.php.