Page MenuHomePhabricator

KeyError in urllib3 since Python 3.5.0b3
Closed, ResolvedPublic

Description

This is probably an Upstream bug, but I just want to have at least a note here. It seems that recently the Python 3.6 builds fail due to urllib3 raising a KeyError (see https://travis-ci.org/xZise/pywikibot-core/builds/71550286#L7189):

======================================================================
ERROR: testQueryApiGetter (tests.wikidataquery_tests.TestApiSlowFunctions)
Test that we can actually retreive data and that caching works.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/xZise/pywikibot-core/tests/wikidataquery_tests.py", line 252, in testQueryApiGetter
    data = w.query(q)
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/data/wikidataquery.py", line 601, in query
    data = self.getDataFromHost(fullQueryString)
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/data/wikidataquery.py", line 563, in getDataFromHost
    resp = http.fetch(url)
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/comms/http.py", line 359, in fetch
    error_handling_callback(request)
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/comms/http.py", line 276, in error_handling_callback
    raise request.data
  File "/home/travis/build/xZise/pywikibot-core/pywikibot/comms/http.py", line 255, in _http_process
    auth=auth, timeout=timeout, verify=True)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/sessions.py", line 465, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/adapters.py", line 337, in send
    conn = self.get_connection(request.url, proxies)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/adapters.py", line 251, in get_connection
    conn = self.poolmanager.connection_from_url(url)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/packages/urllib3/poolmanager.py", line 139, in connection_from_url
    return self.connection_from_host(u.host, port=u.port, scheme=u.scheme)
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/packages/urllib3/poolmanager.py", line 125, in connection_from_host
    self.pools[pool_key] = pool
  File "/home/travis/virtualenv/python3.6-dev/lib/python3.6/site-packages/requests/packages/urllib3/_collections.py", line 66, in __setitem__
    _key, evicted_value = self._container.popitem(last=False)
KeyError: ('https', 'eu.wiktionary.org', 443)

Now considering that we sometimes get errors when using wikidataquery it might be that this is the same type of error but just with different symptomes. And usual requests work, although that error message appears relatively often but it only leads to two failures (search for KeyError: ( and I get more than 100 matches on the Travis build mentioned above).

See also: https://github.com/shazow/urllib3/issues/680 and http://bugs.python.org/issue24667

Event Timeline

XZise claimed this task.
XZise raised the priority of this task from to Needs Triage.
XZise updated the task description. (Show Details)
XZise added a project: Pywikibot.
XZise subscribed.

Just noting that requests refuses to use travis-ci; they have http://ci.kennethreitz.org/ . Their trunk CI doesnt have multiple platforms, but their pull requests CI does have separate builds for 2.6, 2.7, 3.3 and 3.4. No 3.5 let alone 3.6 test builds.

urllib3 is better, with travis-ci enabled and tracking tox platforms py26, py27, py33, py34, pypy and gae (Google App Engine).

Unfortunately I can't really reproduce it locally. When I run python setup.py test it shows that errors at some points but as soon as I execute that test file individually it won't complain. And from looking at that error message it appears to be a caching issue.

XZise removed XZise as the assignee of this task.Jul 18 2015, 7:20 PM
XZise updated the task description. (Show Details)
XZise set Security to None.
XZise renamed this task from KeyError in urllib3 and Python 3.6 to KeyError in urllib3 since Python 3.5.0b3.Jul 23 2015, 1:26 PM
XZise added a project: Upstream.
XZise claimed this task.

I've run a test suite on 3.5.0rc1 and don't get this error anymore.