Page MenuHomePhabricator
Create Task
Maniphest T124287

weblinkchecker.LinkChecker does not work on Python 3
Closed, ResolvedPublic
Actions

Description

There are all sorts of problems.

$ python3 pwb.py weblinkchecker -lang:de -family:wikipedia -page:Landeshauptarchiv_Schwerin -day:0
Retrieving 1 pages from wikipedia:de.


>>> Landeshauptarchiv Schwerin <<<
Contacting server gso.gbv.de to find out its default encoding...
Error retrieving server's default charset. Using ISO 8859-1.
encoding is iso8859-1
WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=13, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59920), raddr=('193.174.240.94', 80)>
  self._sock = None

Exception while processing URL http://gso.gbv.de/CMD?ACT=SRCHA&IKT=1016&SRT=YOP&TRM=SER+Findbücher,+Inventare+und+kleine+Schriften+des+Landeshauptarchivs+Schwerin in page Landeshauptarchiv Schwerin
Exception in thread b'Landeshauptarchiv Schwerin - http://gso.gbv.de/CMD?ACT=SRCHA&IKT=1016&SRT=YOP&TRM=SER+Findb\xc3\xbccher,+Inventare+und+kleine+Schriften+des+Landeshauptarchivs+Schwerin':
Traceback (most recent call last):
  File "./scripts/weblinkchecker.py", line 403, in resolveRedirect
    redirTarget.encode('ascii')
UnicodeEncodeError: 'ascii' codec can't encode character '\xfc' in position 64: ordinal not in range(128)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/threading.py", line 923, in _bootstrap_inner
    self.run()
  File "./scripts/weblinkchecker.py", line 549, in run
    ok, message = linkChecker.check()
  File "./scripts/weblinkchecker.py", line 504, in check
    return redirChecker.check(useHEAD=useHEAD)
  File "./scripts/weblinkchecker.py", line 441, in check
    wasRedirected = self.resolveRedirect(useHEAD=useHEAD)
  File "./scripts/weblinkchecker.py", line 405, in resolveRedirect
    redirTarget = redirTarget.decode(
AttributeError: 'str' object has no attribute 'decode'

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=13, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59948), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=18, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59930), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=25, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59946), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=13, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59960), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=19, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59934), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=24, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59944), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=9, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59914), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=18, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59962), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=17, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59932), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=26, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59950), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=11, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59918), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=32, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59966), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=20, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59940), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=12, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59926), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=28, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59954), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=33, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59968), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=22, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59938), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=15, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59924), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=27, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59952), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=34, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59970), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=21, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59936), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=14, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59922), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=29, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59956), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=35, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59972), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=23, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59942), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=16, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59928), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=36, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59974), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=16, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59990), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=30, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59958), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59916), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=16, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60006), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=30, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60024), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=17, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59992), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=11, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59980), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=23, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60008), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=32, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60026), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=18, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59994), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=9, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59978), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=25, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60012), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=34, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60030), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=19, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59996), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=15, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59988), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=26, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60014), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=36, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60034), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=21, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60000), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=14, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59986), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=24, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60010), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=33, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60028), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=20, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59998), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=12, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59984), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=27, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60016), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=35, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60032), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=22, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60002), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=13, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59982), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=28, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60018), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=38, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60036), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=37, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59976), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=31, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 59964), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=39, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60038), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=12, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60048), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=29, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60020), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60004), raddr=('193.174.240.94', 80)>
  self._sock = None


1 pages read
0 pages written
Execution time: 8 seconds
Read operation time: 8 seconds
Script terminated successfully.
Waiting for remaining 9 threads to finish, please wait...
Exception while processing URL http://mv.juris.de/mv/ArchivG_MV_rahmen.htm in page Landeshauptarchiv Schwerin
Exception in thread b'Landeshauptarchiv Schwerin - http://mv.juris.de/mv/ArchivG_MV_rahmen.htm':
Traceback (most recent call last):
  File "./scripts/weblinkchecker.py", line 441, in check
    wasRedirected = self.resolveRedirect(useHEAD=useHEAD)
  File "./scripts/weblinkchecker.py", line 385, in resolveRedirect
    self.header)
  File "/usr/local/lib/python3.5/http/client.py", line 1030, in request
    self._send_request(method, url, body, headers)
  File "/usr/local/lib/python3.5/http/client.py", line 1068, in _send_request
    self.endheaders(body)
  File "/usr/local/lib/python3.5/http/client.py", line 1026, in endheaders
    self._send_output(message_body)
  File "/usr/local/lib/python3.5/http/client.py", line 866, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.5/http/client.py", line 809, in send
    self.connect()
  File "/usr/local/lib/python3.5/http/client.py", line 785, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/local/lib/python3.5/socket.py", line 689, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/usr/local/lib/python3.5/socket.py", line 728, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -5] No address associated with hostname

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/threading.py", line 923, in _bootstrap_inner
    self.run()
  File "./scripts/weblinkchecker.py", line 549, in run
    ok, message = linkChecker.check()
  File "./scripts/weblinkchecker.py", line 458, in check
    msg = error[1]
TypeError: 'gaierror' object does not support indexing

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=18, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60058), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=12, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60074), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=40, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60040), raddr=('193.174.240.94', 80)>
  self._sock = None

WARNING: /usr/local/lib/python3.5/socket.py:642: ResourceWarning: unclosed <socket.socket fd=10, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.101', 60022), raddr=('193.174.240.94', 80)>
  self._sock = None

Waiting for remaining 7 threads to finish, please wait...
^CInterrupted.
Remaining 6 threads will be killed.
Saving history...

Related Objects
Search...

Event Timeline

jayvdb created this task.Jan 21 2016, 2:30 PM
jayvdb raised the priority of this task from to Needs Triage.
jayvdb updated the task description. (Show Details)
jayvdb added projects: Pywikibot-weblinkchecker.py, Pywikibot.
jayvdb subscribed.
Restricted Application added subscribers: pywikibot-bugs-list, StudiesWorld, Aklapper. · View Herald TranscriptJan 21 2016, 2:30 PM
jayvdb added a subtask: T113140: Convert weblinkchecker to requests.Jan 21 2016, 2:36 PM
jayvdb added a parent task: T60053: Pywikibot Python 3 compatibility (tracking).
jayvdb triaged this task as Lowest priority.Jan 21 2016, 3:19 PM
jayvdb closed subtask T113140: Convert weblinkchecker to requests as Resolved.
AbdealiJK subscribed.Mar 14 2016, 2:20 PM

I was trying to reproduce your error and was unable to. This is the output I got:

$ python3 pwb.py weblinkchecker -family:wikipedia -page:Philosophy -day:0
Retrieving 1 pages from wikipedia:en.


>>> Philosophy <<<

1 pages read
0 pages written
Execution time: 2 seconds
Read operation time: 2 seconds
Script terminated successfully.
Waiting for remaining 49 threads to finish, please wait...
Waiting for remaining 40 threads to finish, please wait...
Waiting for remaining 37 threads to finish, please wait...
Waiting for remaining 35 threads to finish, please wait...
Waiting for remaining 33 threads to finish, please wait...
Waiting for remaining 33 threads to finish, please wait...
Waiting for remaining 33 threads to finish, please wait...
Waiting for remaining 33 threads to finish, please wait...
Waiting for remaining 33 threads to finish, please wait...
Waiting for remaining 33 threads to finish, please wait...
Waiting for remaining 33 threads to finish, please wait...
Waiting for remaining 33 threads to finish, please wait...
Waiting for remaining 33 threads to finish, please wait...
Waiting for remaining 33 threads to finish, please wait...
Waiting for remaining 33 threads to finish, please wait...
Waiting for remaining 29 threads to finish, please wait...
Waiting for remaining 19 threads to finish, please wait...
Waiting for remaining 8 threads to finish, please wait...
Waiting for remaining 7 threads to finish, please wait...
Waiting for remaining 7 threads to finish, please wait...
Exception while processing URL http://machaut.uchicago.edu/?resource=Webster%27s&word=epistemology&use1913=on in page Philosophy
Exception in thread b'Philosophy - http://machaut.uchicago.edu/?resource=Webster%27s&word=epistemology&use1913=on':
Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 137, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/util/connection.py", line 67, in create_connection
    for res in socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/socket.py", line 732, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
    body=body, headers=headers)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 353, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 1083, in request
    self._send_request(method, url, body, headers)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 1128, in _send_request
    self.endheaders(body)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 1079, in endheaders
    self._send_output(message_body)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 911, in _send_output
    self.send(msg)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 854, in send
    self.connect()
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 162, in connect
    conn = self._new_conn()
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 146, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
requests.packages.urllib3.exceptions.NewConnectionError: <requests.packages.urllib3.connection.HTTPConnection object at 0xb4fafeac>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/adapters.py", line 376, in send
    timeout=timeout
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 609, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/util/retry.py", line 273, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
requests.packages.urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='machaut.uchicago.edu', port=80): Max retries exceeded with url: /?resource=Webster%27s&word=epistemology&use1913=on (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0xb4fafeac>: Failed to establish a new connection: [Errno -2] Name or service not known',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "./scripts/weblinkchecker.py", line 568, in run
    r = requests.get(self.url, headers=header, timeout=timeout)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/api.py", line 67, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/adapters.py", line 437, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='machaut.uchicago.edu', port=80): Max retries exceeded with url: /?resource=Webster%27s&word=epistemology&use1913=on (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0xb4fafeac>: Failed to establish a new connection: [Errno -2] Name or service not known',))

Waiting for remaining 5 threads to finish, please wait...
Exception while processing URL http://www.oed.com.ezproxy.uky.edu/view/Entry/142505?rskey=uk0M8u&result=1 in page Philosophy
Exception in thread b'Philosophy - http://www.oed.com.ezproxy.uky.edu/view/Entry/142505?rskey=uk0M8u&result=1':
Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
    body=body, headers=headers)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 345, in _make_request
    self._validate_conn(conn)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 784, in _validate_conn
    conn.connect()
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 252, in connect
    ssl_version=resolved_ssl_version)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/util/ssl_.py", line 305, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/ssl.py", line 376, in wrap_socket
    _context=self)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/ssl.py", line 747, in __init__
    self.do_handshake()
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/ssl.py", line 983, in do_handshake
    self._sslobj.do_handshake()
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/ssl.py", line 628, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/adapters.py", line 376, in send
    timeout=timeout
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 588, in urlopen
    raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "./scripts/weblinkchecker.py", line 568, in run
    r = requests.get(self.url, headers=header, timeout=timeout)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/api.py", line 67, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/sessions.py", line 597, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/sessions.py", line 597, in <listcomp>
    history = [resp for resp in gen] if allow_redirects else []
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/sessions.py", line 195, in resolve_redirects
    **adapter_kwargs
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/adapters.py", line 447, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)

Waiting for remaining 3 threads to finish, please wait...
Waiting for remaining 3 threads to finish, please wait...
Waiting for remaining 3 threads to finish, please wait...
Waiting for remaining 3 threads to finish, please wait...
Waiting for remaining 1 threads to finish, please wait...
Waiting for remaining 1 threads to finish, please wait...
Waiting for remaining 1 threads to finish, please wait...
Waiting for remaining 1 threads to finish, please wait...
Waiting for remaining 1 threads to finish, please wait...
Exception while processing URL http://www.journals.cambridge.org/phi in page Philosophy
Exception in thread b'Philosophy - http://www.journals.cambridge.org/phi':
Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 376, in _make_request
    httplib_response = conn.getresponse(buffering=True)
TypeError: getresponse() got an unexpected keyword argument 'buffering'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 378, in _make_request
    httplib_response = conn.getresponse()
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 1174, in getresponse
    response.begin()
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 282, in begin
    version, status, reason = self._read_status()
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 243, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/socket.py", line 575, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/adapters.py", line 376, in send
    timeout=timeout
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 609, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/util/retry.py", line 247, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/packages/six.py", line 310, in reraise
    raise value
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
    body=body, headers=headers)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 380, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 308, in _raise_timeout
    raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
requests.packages.urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='www.journals.cambridge.org', port=80): Read timed out. (read timeout=30)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "./scripts/weblinkchecker.py", line 568, in run
    r = requests.get(self.url, headers=header, timeout=timeout)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/api.py", line 67, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/adapters.py", line 449, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='www.journals.cambridge.org', port=80): Read timed out. (read timeout=30)

Saving history...

These aren't issues with python3, but with weblinkchecker.
It only seems to check for requests.exceptions.InvalidURL whereas we should also be checking for requests.exceptions.SSLError and requests.exceptions.ReadTimeout

In fact, we should be doing requests.exceptions.RequestException - the error from which all request exceptions inherit.
But that should be another issue in itself.

@jayvdb, do you still get this issue ? If yes, could you elaborate on steps to reproduce it ?

jayvdb added a comment.Mar 14 2016, 3:06 PM

Did you try to reproduce using python3 pwb.py weblinkchecker -lang:de -family:wikipedia -page:Landeshauptarchiv_Schwerin -day:0 , which is https://de.wikipedia.org/wiki/Landeshauptarchiv_Schwerin . The problem was the http://gso.gbv.de/... link in the page, which exercises the problematic chunk of code.

I believe there have been some improvements to weblinkchecker.py since this ticket was raised, so this task might be invalid now.

AbdealiJK added a comment.Mar 14 2016, 3:10 PM

@jayvdb I forgot to mention:

I did try with python3 pwb.py weblinkchecker -lang:de -family:wikipedia -page:Landeshauptarchiv_Schwerin -day:0 first, and got confused due to the language.

I got one link which gave requests.exceptions.ConnectionError: but nothing like your earlier description. So I think this ought to be closed.

PS : requests.exceptions.ConnectionError also inherits from requests.exceptions.RequestException

Output:

$ python3 pwb.py weblinkchecker -lang:de -family:wikipedia -page:Landeshauptarchiv_Schwerin -day:0
Retrieving 1 pages from wikipedia:de.


>>> Landeshauptarchiv Schwerin <<<

1 pages read
0 pages written
Execution time: 1 seconds
Read operation time: 1 seconds
Script terminated successfully.
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 25 threads to finish, please wait...
Waiting for remaining 17 threads to finish, please wait...
Waiting for remaining 3 threads to finish, please wait...
Waiting for remaining 3 threads to finish, please wait...
Waiting for remaining 3 threads to finish, please wait...
Exception while processing URL http://mv.juris.de/mv/ArchivG_MV_rahmen.htm in page Landeshauptarchiv Schwerin
Waiting for remaining 3 threads to finish, please wait...
Exception in thread b'Landeshauptarchiv Schwerin - http://mv.juris.de/mv/ArchivG_MV_rahmen.htm':
Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 137, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/util/connection.py", line 67, in create_connection
    for res in socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/socket.py", line 732, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
    body=body, headers=headers)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 353, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 1083, in request
    self._send_request(method, url, body, headers)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 1128, in _send_request
    self.endheaders(body)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 1079, in endheaders
    self._send_output(message_body)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 911, in _send_output
    self.send(msg)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/http/client.py", line 854, in send
    self.connect()
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 162, in connect
    conn = self._new_conn()
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connection.py", line 146, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
requests.packages.urllib3.exceptions.NewConnectionError: <requests.packages.urllib3.connection.HTTPConnection object at 0xb58404ec>: Failed to establish a new connection: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/adapters.py", line 376, in send
    timeout=timeout
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 609, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/packages/urllib3/util/retry.py", line 273, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
requests.packages.urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='mv.juris.de', port=80): Max retries exceeded with url: /mv/ArchivG_MV_rahmen.htm (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0xb58404ec>: Failed to establish a new connection: [Errno -2] Name or service not known',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "./scripts/weblinkchecker.py", line 568, in run
    r = requests.get(self.url, headers=header, timeout=timeout)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/api.py", line 67, in get
    return request('get', url, params=params, **kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/home/ajk/.pyenv/versions/3.5.1/lib/python3.5/site-packages/requests/adapters.py", line 437, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='mv.juris.de', port=80): Max retries exceeded with url: /mv/ArchivG_MV_rahmen.htm (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0xb58404ec>: Failed to establish a new connection: [Errno -2] Name or service not known',))

Waiting for remaining 2 threads to finish, please wait...
Saving history...
AbdealiJK mentioned this in T129611: [GSoC 2016 Proposal] Port catimages.py to pywikibot-core.Mar 15 2016, 3:01 AM
Xqt added a subtask: T135302: wikidataquery_tests.TestApiSlowFunctions.testQueryApiGetter fails for python 3.4.May 14 2016, 9:14 AM
Multichill closed subtask T135302: wikidataquery_tests.TestApiSlowFunctions.testQueryApiGetter fails for python 3.4 as Declined.May 20 2017, 1:16 PM
Xqt mentioned this in T183696: site_tests.TestDeprecatedEditTokenFunctions.test_token failes sometimes with AssertionError due to unclosed socket.socket ResourceWarning.Dec 26 2017, 11:26 AM
Dvorapa closed this task as Resolved.EditedJan 31 2018, 5:04 PM
Dvorapa claimed this task.
Dvorapa subscribed.

No longer reproducible with that gso.gbv.de link, probably fixed

I reproduced this on the old revision (current to date this task was created) of discussed page with current version of pwb, output was correct. @jayvdb you can try this again and reopen if still broken

Dvorapa added a comment.Jan 31 2018, 5:07 PM
This comment was removed by Dvorapa.
Dvorapa mentioned this in T186131: Make weblinkchecker.py handle SSLError, ReadTimeout, ConnectionError, TooManyRedirects and others.Jan 31 2018, 5:54 PM
Dvorapa removed Dvorapa as the assignee of this task.Mar 24 2019, 8:35 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL