Strict SSL certificate checking
Closed, DeclinedPublic
Actions

Assigned To

None

Authored By

	saper
	May 5 2015, 1:03 AM

Description

ssl module may need strict checking:

`
m> env PATH= /usr/local/bin/python2.7 version.py                                                                                                              
Retreving commit log from https://tools.wmflabs.org/pywikibot/gitlog.txt
Traceback (most recent call last):
  File "version.py", line 20, in <module>
    pywikibot.output('Pywikibot: %s' % getversion())
  File "/home/saper/wikipedia/compat/pywikibot/version.py", line 46, in getversion
    data = dict(getversiondict())  # copy dict to prevent changes in 'chache'
  File "/home/saper/wikipedia/compat/pywikibot/version.py", line 70, in getversiondict
    (tag, rev, date, hsh) = getversion_git(_program_dir)
  File "/home/saper/wikipedia/compat/pywikibot/version.py", line 192, in getversion_git
    rev, date = getversion_git_windows(hsh, path)
  File "/home/saper/wikipedia/compat/pywikibot/version.py", line 111, in getversion_git_windows
    ff = urllib2.urlopen(url).read().splitlines()
  File "/usr/local/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/local/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/local/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python2.7/urllib2.py", line 1240, in https_open
    context=self._context)
  File "/usr/local/lib/python2.7/urllib2.py", line 1197, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)>
`

Workaround: list of certificate authorities you trust SSL_CERT_FILE

I think SSL certificates could/should be supplied attached to the family files, not sure where else do we connect to (tools.wmflabs.org, anything else?...)

Related Objects

Mentioned In: T98107: Ability to disable Pywikibot SCM version detection completely
T98106: writeToCommandLogFile() should be optional

Event Timeline

saper created this task.May 5 2015, 1:03 AM

saper raised the priority of this task from to Medium.

saper updated the task description. (Show Details)

saper subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 5 2015, 1:03 AM

saper added a project: Pywikibot-General.May 5 2015, 1:04 AM

saper set Security to None.

Restricted Application added a subscriber: Unknown Object (MLST). · View Herald TranscriptMay 5 2015, 1:04 AM

saper mentioned this in T98106: writeToCommandLogFile() should be optional.May 5 2015, 1:12 AM

saper mentioned this in T98107: Ability to disable Pywikibot SCM version detection completely.May 5 2015, 1:25 AM

are you experiencing this when using pywikibot-compat with Wikimedia wikis, or other wikis?

We've seen very similar problems with Wikimedia wikis, when the server had old certificates, and the problem was fixed by installing new root certificates.

Either way, there is a now a per-family flag in core to disable ssl verification, which avoids this problem. I do like the solution of allowing certificates to be attached to each family , however that is almost identical to disabling ssl verification, unless the certificates are properly maintained (i.e. checks against certificate revocation lists, etc)

Yes, I am accessing Wikimedia wikis. I get this error trying to access the API (e.g. login) but also connecting to https://tools.wmflabs.org/

This is with all SSL connections with my Python (2.7.9 from FreeBSD ports), so it is not in that sense Wikimedia-specific.

I think I'd rather have one CA certificate blob per site (or group of sites) instead of blindly trusting some random CA collection off the Internet. I am sure that at least for Wikimedia wikis the CA certificate will be properly maintained.

jayvdb edited projects, added Pywikibot-network; removed Pywikibot-General.May 6 2015, 3:07 AM

I see this backtrace is 'compat', which isnt maintained.

Do you also experience this problem in core? I have rewritten those lines in core.

We have switched 'core' to using 'requests', so it is now possible to specify a bundle in an environment variable, and has lots of other options for adding trusted cert bundles.

Regarding properly maintained certs in family files, the problem is release management. If we release pywikibot-2.0 with a certificate for wikipedia.org , the family class will fail if that certificate is invalidated, which will mean we need to release a new version of pywikibot. I do not see the benefit in storing certificates for normal wikis which use a normal CA. Storing a certificate could be useful for wikis which self-sign their certificate, etc.

jayvdb lowered the priority of this task from Medium to Lowest.Jun 3 2015, 2:57 AM

Pywikibot has two versions: Compat and Core. This task was filed about the older version, called Pywikibot-compat, which is not under active development anymore. Hence I'm lowering the priority of this task to reflect the reality. Unfortunately, the Pywikibot team does not have the manpower to retest every single bug report / feature request against the (maintained) Pywikibot code base. Furthermore, the code base of Pywikibot-Compat has changed a lot compared to the code base of Pywikibot-Core so there is a chance that the problem described in this task might not exist anymore. Please help: Unfortunately manpower is limited and does not allow testing every single reported task again. If you have time and interest in Pywikibot, please upgrade to Pywikibot-Core and add a comment to this task if the problem in this task still happens in Pywikibot-Core (or directly edit the task by removing the Pywikibot-compat project and adding the Pywikibot project to this task). To learn more about Pywikibot and to get involved in its development, please check out https://www.mediawiki.org/wiki/Manual:Pywikibot/Development Thank you for your understanding.

Won't fix in compat

Strict SSL certificate checkingClosed, DeclinedPublicActions

Description

Related Objects

Event Timeline

Strict SSL certificate checking
Closed, DeclinedPublic
Actions