Page MenuHomePhabricator

site detection test fails for proofwiki on Travis precise images before May 2015
Closed, ResolvedPublic

Description

Proofwiki uses TLSv1.2 only, which is explicitly rejected by early Ubuntu precise images. Ubuntu fixed this in May 2015, but Travis is using images before that.
The workaround for Travis is to use newer ssl libraries on the Ubuntu precise image.

3672FAIL: test_detect_site (tests.site_detect_tests.SiteDetectionTestCase)
3673Test detection of MediaWiki sites.
3674----------------------------------------------------------------------
3675Traceback (most recent call last):
3676  File "/home/travis/build/wikimedia/pywikibot-core/tests/site_detect_tests.py", line 183, in test_detect_site
3677    self.assertAllPass()
3678  File "/home/travis/build/wikimedia/pywikibot-core/tests/site_detect_tests.py", line 115, in assertAllPass
3679    self.assertEqual(set(self.passes), set(self.all) - set(self.skips))
3680AssertionError: Items in the second set but not the first:
3681'http://www.proofwiki.org/wiki/$1'

https://travis-ci.org/wikimedia/pywikibot-core/jobs/128508437

Event Timeline

Additional information

2981test_detect_site (tests.site_detect_tests.SiteDetectionTestCase)
2982Test detection of MediaWiki sites. ... failure http://www.proofwiki.org/wiki/$1 on <class 'requests.exceptions.ConnectionError'>: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
2983Out of 9 sites, 8 tests passed, 0 tests failed, 0 tests skiped and 1 tests raised an error
2984Results:
2985PASS http://www.hrwiki.org/index.php/$1                : MWSite("http://www.hrwiki.org/wiki/Main_Page")
2986PASS http://www.ck-wissen.de/ckwiki/index.php?title=$1 : MWSite("http://www.ck-wissen.de/ckwiki/index.php?title=Hauptseite")
2987PASS http://en.citizendium.org/wiki/$1                 : MWSite("http://en.citizendium.org/wiki/")
2988PASS http://www.lojban.org/tiki/tiki-index.php?page=$1 : MWSite("https://mw.lojban.org/papri/Lojban")
2989PASS http://www.wikichristian.org/index.php?title=$1   : MWSite("http://www.wikichristian.org/wiki/en/index.php?title=Main_Page")
2990PASS https://en.wikifur.com/wiki/$1                    : MWSite("https://en.wikifur.com/wiki/WikiFur_Furry_Central")
2991PASS http://bluwiki.com/go/$1                          : MWSite("http://bluwiki.com/go/Main_Page")
2992PASS http://kino.skripov.com/index.php/$1              : MWSite("http://www.kino-wiki.com/wiki/index.php?/")
2993ERR  http://www.proofwiki.org/wiki/$1                  : ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
2994FAIL

Change 287419 had a related patch set uploaded (by Xqt):
[bugfix] remove proofwiki.org from test suite

https://gerrit.wikimedia.org/r/287419

Xqt triaged this task as Medium priority.

The patch says "SiteDetectionTestCase.test_detect_site() failes very often for proofwiki.org"

How frequently?
Do we know why it is failing?

https://proofwiki.org/wiki/Main_Page is online now.

Refer the last hundreds of Travis tests. They always failed due to a connection test for this site. We could either spend a lot of time for debugging why the connection is reseted all the time or spend some amount of time to enable that horrible code part to silently ignore that connection error as expected failure and maybe investigate into that matter later. Or more preferable deactivate this site test to get a clean test now which ensures that we can focus on more important bugs (and maybe if we have some remaining time debug this site related bug further).

I changed the patch. It skips now for requests.ConnectionError.

jayvdb raised the priority of this task from Medium to High.May 17 2016, 6:28 PM
jayvdb added a project: Pywikibot-network.

There is definitely a real bug here.

I can reliably reproduce the problem with just wget on Travis, and reliably avoid the problem by using trusty with sudo: required

https://travis-ci.org/jayvdb/my-ci-test/builds/130797731 (the first job is trusty with sudo: required, the others are precise)

There is something different between these images, and whatever it is can be also affecting a real user.
If I cant figure it out tomorrow, I'll submit a patch to disable proofwiki only on travis, which is the only place where we have seen the problem occurring.

proofwiki doesnt fail on circleci which is Python 2.7 on Ubuntu precise and trusty.

jayvdb renamed this task from site detection test fails for test_detect_site SiteDetectionTestCase.test_detect_site to site detection test fails for proofwiki on Travis precise.May 19 2016, 4:32 AM

After a bunch of digging and testing, I believe that the reason that this fails is because the website www.proofwiki.org only supports TLS v1.2
I tried https://travis-ci.org/AbdealiJK/my-ci-test/builds/131360417 where I compared the working of www.proofwiki.org and en.wikifur.com. The latter works fine on all distros. Hence, I tried to compare the prototcols better and found:

In the ssllabs website, you can see that proofwiki supports only TLSv1.2 while wikifur supports TLS v1, v1.1, v1.2. There are many browsers that do not support TLS v1.2 (seen in the "Handshake Simulation" section) - including OpenSSL v0.9 (v1.0+ supports it).

I found a similar issue here where curl also cannot download from a site which has only TLSv1.2 on travis: https://github.com/travis-ci/travis-ci/issues/4757

So, this is a known issue with travis.
I think either proofwiki should reduce their "secureness" and allow TLS of lower versions or the website won't run correctly on older computers having old software.

What does this mean for us?

I think this is not a pywikibot issue, and that we should remove the proofwiki test because it's not a reliable test (as it won't work on a bunch of platforms).

If we still want the test:
A good idea would be to check if www.proofwiki.org can be downloaded (using requests directly?), and if it can't, we should skip the test.

Possibly there should be a method to check if requests supports TLS v1.2 and skip based on that, but I can't find anything that can check that :(

Yea. Ive found the same and am adding a job that works around the problem.

I've managed to get the trusty container build to connect with requests, by upgrading everything.
https://travis-ci.org/jayvdb/my-ci-test/jobs/131433871#L1708
(and wget latter in the log shows the underlying problem nicely)

Next step is selectively upgrading specific packages to find which one solves the problem - probably an ssl related package.

Change 289819 had a related patch set uploaded (by John Vandenberg):
Detect connection refused for TLSv1.2 only sites

https://gerrit.wikimedia.org/r/289819

Change 287419 abandoned by Xqt:
[bugfix] Skip proofwiki.org test when ConnectionError occurs

Reason:
see Icb374767daf963ee31eab4b16fa8a4f86b57afd4

https://gerrit.wikimedia.org/r/287419

As mentioned on https://github.com/travis-ci/travis-ci/issues/4757#issuecomment-220820490 , the 'solution' is

addons:
  apt:
    sources:
      - debian-sid
    packages:
      - libssl1.0.0

However, using debian-sid conflicts with using graphviz and python-ipaddr ; both cause APT to error, as those packages on debian-sid pull in many additional dependencies that conflict with ubuntu precise packages.

Change 290146 had a related patch set uploaded (by John Vandenberg):
Travis workaround for TLSv1.2 on precise container

https://gerrit.wikimedia.org/r/290146

python-ipaddr is only useful on the 2.7_with_system_site_packages jobs, and graphviz is only used by interwiki graph tests, and only one line looses coverage with the above patch, because those interwiki graph tests are skipped unless PYSETUP_TEST_EXTRAS=1

https://codecov.io/gh/jayvdb/pywikibot-core/src/16856bec551e120b733ea141b868d1f009e2c13b/pywikibot/interwiki_graph.py#L60

Change 290146 merged by jenkins-bot:
Travis workaround for TLSv1.2 on precise container

https://gerrit.wikimedia.org/r/290146

jayvdb renamed this task from site detection test fails for proofwiki on Travis precise to site detection test fails for proofwiki on Travis precise images before May 2015.May 27 2016, 1:50 AM
jayvdb lowered the priority of this task from High to Low.
jayvdb removed a project: Patch-For-Review.
jayvdb updated the task description. (Show Details)

Just noting that the sudoless image is still
Build image provisioning date and time: Thu Feb 5 15:09:33 UTC 2015

Magul subscribed.

I looks that there is nothing else to be done here. Test for http://www.proofwiki.org/wiki/ works. I'm closing it.

Change 289819 abandoned by Xqt:
Detect connection refused for TLSv1.2 only sites

https://gerrit.wikimedia.org/r/289819

Change 662900 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [tests] Remove libssl1.0.0 dependency

https://gerrit.wikimedia.org/r/662900

Change 662900 merged by jenkins-bot:
[pywikibot/core@master] [tests] Remove libssl1.0.0 dependency

https://gerrit.wikimedia.org/r/662900