Page MenuHomePhabricator

KeyError in urllib3 since Python 3.5.0b3
Closed, ResolvedPublic

Description

This is probably an Upstream bug, but I just want to have at least a note here. It seems that recently the Python 3.6 builds fail due to urllib3 raising a KeyError (see https://travis-ci.org/xZise/pywikibot-core/builds/71550286#L7189):

======================================================================
ERROR: testQueryApiGetter (tests.wikidataquery_tests.TestApiSlowFunctions)
Test that we can actually retreive data and that caching works.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/xZise/pywikibot-core/tests/wikidataquery_tests.py", line 252, in testQueryApiGetter
    data = w.query(q)
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/data/wikidataquery.py", line 601, in query
    data = self.getDataFromHost(fullQueryString)
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/data/wikidataquery.py", line 563, in getDataFromHost
    resp = http.fetch(url)
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/comms/http.py", line 359, in fetch
    error_handling_callback(request)
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/comms/http.py", line 276, in error_handling_callback
    raise request.data
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/comms/http.py", line 255, in _http_process
    auth=auth, timeout=timeout, verify=True)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/sessions.py", line 465, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/adapters.py", line 337, in send
    conn = self.get_connection(request.url, proxies)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/adapters.py", line 251, in get_connection
    conn = self.poolmanager.connection_from_url(url)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/packages/urllib3/poolmanager.py", line 139, in connection_from_url
    return self.connection_from_host(u.host, port=u.port, scheme=u.scheme)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/packages/urllib3/poolmanager.py", line 125, in connection_from_host
    self.pools[pool_key] = pool
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/packages/urllib3/_collections.py", line 66, in __setitem__
    _key, evicted_value = self._container.popitem(last=False)
KeyError: ('https', 'eu.wiktionary.org', 443)

Now considering that we sometimes get errors when using wikidataquery it might be that this is the same type of error but just with different symptomes. And usual requests work, although that error message appears relatively often but it only leads to two failures (search for KeyError: ( and I get more than 100 matches on the Travis build mentioned above).

See also: https://github.com/shazow/urllib3/issues/680 and http://bugs.python.org/issue24667

Event Timeline

XZise created this task.Jul 18 2015, 2:42 PM
XZise claimed this task.
XZise raised the priority of this task from to Needs Triage.
XZise updated the task description. (Show Details)
XZise added a project: Pywikibot.
XZise added a subscriber: XZise.
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptJul 18 2015, 2:42 PM
jayvdb added a subscriber: jayvdb.Jul 18 2015, 5:44 PM

Just noting that requests refuses to use travis-ci; they have http://ci.kennethreitz.org/ . Their trunk CI doesnt have multiple platforms, but their pull requests CI does have separate builds for 2.6, 2.7, 3.3 and 3.4. No 3.5 let alone 3.6 test builds.

urllib3 is better, with travis-ci enabled and tracking tox platforms py26, py27, py33, py34, pypy and gae (Google App Engine).

XZise added a comment.Jul 18 2015, 7:13 PM

Unfortunately I can't really reproduce it locally. When I run python setup.py test it shows that errors at some points but as soon as I execute that test file individually it won't complain. And from looking at that error message it appears to be a caching issue.

XZise removed XZise as the assignee of this task.Jul 18 2015, 7:20 PM
XZise updated the task description. (Show Details)
XZise set Security to None.
XZise updated the task description. (Show Details)Jul 19 2015, 10:41 AM
XZise renamed this task from KeyError in urllib3 and Python 3.6 to KeyError in urllib3 since Python 3.5.0b3.Jul 23 2015, 1:26 PM
XZise added a project: Upstream.
jayvdb moved this task from Backlog to Patch merged upstream on the Upstream board.Aug 5 2015, 4:31 PM
jayvdb moved this task from Patch merged upstream to Patch proposed upstream on the Upstream board.
jayvdb moved this task from Patch proposed upstream to Reported Upstream on the Upstream board.
XZise closed this task as Resolved.Aug 18 2015, 11:28 PM
XZise claimed this task.

I've run a test suite on 3.5.0rc1 and don't get this error anymore.