Page MenuHomePhabricator

Add timeouts to crawler calls to external URLs
Closed, ResolvedPublic

Description

Security Review Summary - T273020 - 2021-08-09
Last commit reviewed: d9e475d1ff13

[...snip...]

FileLine(s)IssueRisk
apps/crawler/tasks.py168By default, 'requests' calls wait until the connection is closed. This means a 'requests' call without a timeout will hang the program if a response is never received. Consider setting a timeout for all 'requests'. Medium

https://docs.python-requests.org/en/master/user/advanced/#timeouts

Event Timeline

bd808 triaged this task as High priority.Aug 10 2021, 3:56 PM
bd808 created this task.

Change 711206 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[wikimedia/toolhub@main] crawler: Add tcp connect and read timeouts

https://gerrit.wikimedia.org/r/711206

bd808 moved this task from Backlog to Review on the Toolhub board.

Change 711206 merged by jenkins-bot:

[wikimedia/toolhub@main] crawler: Add tcp connect and read timeouts

https://gerrit.wikimedia.org/r/711206