Page MenuHomePhabricator

requests emits InsecurePlatformWarning
Closed, ResolvedPublic

Description

The switch from httplib2 to requests has resulted in a lot of InsecurePlatformWarnings, appearing in ordinary usage but especially in the test suite.

Also it appears that this causes additional problems on Python 2.6 within the test framework's deprecation system.

Event Timeline

jayvdb raised the priority of this task from to High.
jayvdb updated the task description. (Show Details)
jayvdb added subscribers: XZise, gerritbot, Aklapper and 2 others.

Change 220388 had a related patch set uploaded (by John Vandenberg):
Add extra dependencies to fix InsecurePlatformWarning on Python <2.7.9

https://gerrit.wikimedia.org/r/220388

If possible, we should be adding the 'security'(or betterssl) extra when we specify the requests dependency, as that should add all the necessary dependencies. They changed the name fairly recently (https://github.com/kennethreitz/requests/commit/958845ae35cee1fa15acc14b5691d787e8bed9bb), but that shouldnt be a problem unless we specify minimum versions of requests.

If requests doesnt add all the appropriate dependencies, those optional dependencies should be added to the requests setup.py, tracking the upstream problem here.

jayvdb raised the priority of this task from High to Unbreak Now!.
jayvdb set Security to None.

It's the other way around. I'm not sure if we know but at least I guess the errors in T102365 are happening because of the warning shown here. And it then complains that the warning is not happening wherever the deprecated method is executed but somewhere else (although it actually checked the wrong warning). So when you get the warning here fixed so that it doesn't get emitted it wouldn't test the InsecurePlatformWarning and it should actually compare the deprecation warning.

@VcamX , https://gerrit.wikimedia.org/r/220388 needs to be polished - when that is merged, most of the T102365 problems will disappear.

@XZise I agree with you.
I think @Legoktm's patch could fix InsecurePlatformWarning. But it also emits two new DeprecationWarning:

/Users/VcamX/Documents/workspace/core/env/lib/python2.6/site-packages/cryptography/__init__.py:25: DeprecationWarning: Python 2.6 is no longer supported by the Python core team, please upgrade your Python.
  DeprecationWarning
WARNING: /Users/VcamX/Documents/workspace/core/env/lib/python2.6/site-packages/requests/packages/urllib3/contrib/pyopenssl.py:198: DeprecationWarning: unicode for buf is no longer accepted, use bytes
  return self.connection.send(data)

The cryptography's deprecation warning could be depressed by using its 0.8.2 version (the latest is 0.9.3) which doesn't warn that we're using 2.6. But I don't think it's a good solution. It has many changes in its latest version: CHANGELOG

I think the second deprecation warnnig is caused by sending unicode data: https://github.com/pyca/pyopenssl/blob/master/OpenSSL/SSL.py#L1261, https://github.com/pyca/pyopenssl/blob/master/OpenSSL/_util.py#L107
We may encode data using utf-8 by ourselves in _http_process of comms/http.py. This is unconfirmed. I'm working on it.

I disagree that we should just blindly encode data using UTF-8. Instead we should make sure that our requests just contains binary data and no unicode strings. @valhallasw has two patches which should make sure of that. Maybe you could test it with that.

Regarding the other warning I'm not sure what the best handling there is. Some OS only support Python 2.6 but these might provide packages which don't emit that warning.

I disagree that we should just blindly encode data using UTF-8. Instead we should make sure that our requests just contains binary data and no unicode strings. @valhallasw has two patches which should make sure of that. Maybe you could test it with that.

Regarding the other warning I'm not sure what the best handling there is. Some OS only support Python 2.6 but these might provide packages which don't emit that warning.

I think only RHEL is a 'supported' Python 2.6 platform. So wrt to the Travis-CI problems on Python 2.6, we could try to emulate RHEL Python 2.6 by using the versions provided by RHEL at:
http://ftp.redhat.com/pub/redhat/linux/enterprise/6Server/en/os/SRPMS/

i.e. python-ipaddr-2.1.9 (see T105443), python-crypto-2.0.1 , python-pyasn1-0.0.12a, but I do not see a python-requests in that list.

Change 220388 merged by jenkins-bot:
[FIX] InsecurePlatformWarning on Python <2.7.9

https://gerrit.wikimedia.org/r/220388

@VcamX: you added in PS7 of the requests patch an additional condition for our tests. I guess this not necessary anymore (and it would test that the patch works)?

@XZise Okay, I'll send a new patch for that.

Change 225666 had a related patch set uploaded (by John Vandenberg):
remove filename deprecation workaround

https://gerrit.wikimedia.org/r/225666

Change 225666 merged by jenkins-bot:
remove filename deprecation workaround

https://gerrit.wikimedia.org/r/225666