Page MenuHomePhabricator

Threads killed after 30 seconds without reporting URLs that connection failed
Closed, ResolvedPublic

Description

Once all of the threads are started, the main() loops for 30 seconds for all threads to finish, and then it kills all threads still running.

The thread can be running for more than 30 seconds because the connection has not yet started. When this happens, the thread is killed and the URL isnt included in the deadlinks dataset.

The 30 seconds should be configurable, and when that time has expired the links should be presumed dead.

Event Timeline

jayvdb raised the priority of this task from to Needs Triage.
jayvdb updated the task description. (Show Details)
jayvdb subscribed.
jayvdb triaged this task as High priority.Jan 21 2016, 3:22 PM

Change 789854 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [IMPR] Do not kill threads in weblinkchecker.py

https://gerrit.wikimedia.org/r/789854

Change 789854 merged by jenkins-bot:

[pywikibot/core@master] [IMPR] Do not kill threads in weblinkchecker.py

https://gerrit.wikimedia.org/r/789854