Page MenuHomePhabricator

Python 3.5/3.6 compatibility
Closed, ResolvedPublic

Description

PEP 494: Python 3.6 is slated for release in late 2016

Travis-CI's 'latest' target switched to 3.6 alphas, and broke Pywikibot builds. We can expect many more breakages wrt 3.6 as the devs cut and hack.

Python 3.5 is not (yet?) listed as a target on http://docs.travis-ci.com/user/languages/python/ ; Reported Upstream as https://github.com/travis-ci/travis-ci/issues/4437

Details

Related Gerrit Patches:

Event Timeline

jayvdb created this task.Jul 18 2015, 5:09 PM
jayvdb raised the priority of this task from to Medium.
jayvdb updated the task description. (Show Details)
jayvdb added a project: Pywikibot.
jayvdb added subscribers: jayvdb, XZise.
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptJul 18 2015, 5:09 PM

There is travis-build # 477 which will provide a '3.5' target.

jayvdb updated the task description. (Show Details)Jul 18 2015, 5:31 PM
jayvdb added a project: Upstream.
jayvdb set Security to None.
XZise added a comment.Jul 18 2015, 7:11 PM

Why do you expect many more breakages with 3.6 considering that we had few (if any) breakages with 3.5. Do they plan to remove a lot of stuff?

Why do you expect many more breakages with 3.6 considering that we had few (if any) breakages with 3.5. Do they plan to remove a lot of stuff?

There are very few clues on https://docs.python.org/dev/whatsnew/3.6.html
Python 3.5 broke mwparserfromhell, due to its setup.py. And it is library dependencies where I expect problems will arise, as they are not doing builds against these newer versions of Python, and our tests dont exercise all of their code.

Change 225684 had a related patch set uploaded (by John Vandenberg):
Set nightly to allowed_failure

https://gerrit.wikimedia.org/r/225684

Change 225684 merged by jenkins-bot:
Set nightly to allowed_failure

https://gerrit.wikimedia.org/r/225684

jayvdb moved this task from Backlog to Reported Upstream on the Upstream board.Jul 21 2015, 2:50 AM

The 3.5/3.6 target problem may have been resolved with https://github.com/travis-ci/travis-build/pull/477 .

http://docs.travis-ci.com/user/languages/python/#On-demand-installations :
"For a limited number of Python development releases, on-demand installation is available.
Currently, these are: 3.5.0b2, 3.5.0b3, and 3.5-dev (built nightly)."

So I guess we want 3.5-dev

XZise added a comment.Jul 23 2015, 1:24 PM

Do we ask them to install/enable it for us? AFAIK do we have both problems we already encountered in 3.6 also in 3.5. So if we enable 3.5 builds I guess we need to make them expected failures.

XZise renamed this task from Python 3.6 compatibility to Python 3.5/3.6 compatibility.Jul 23 2015, 1:25 PM

Also a general question: Do we want a tracking bug for each version? Or is this the "next Python released(es)" tracking bug?

Do we ask them to install/enable it for us? AFAIK do we have both problems we already encountered in 3.6 also in 3.5. So if we enable 3.5 builds I guess we need to make them expected failures.

I believe we simply use '3.5-dev' as the python version.
Yes, we'd need to add it as an expected failure due to the known bugs.

Also a general question: Do we want a tracking bug for each version? Or is this the "next Python released(es)" tracking bug?

Originally intended to be Python 3.6, but since the Python 3.6 problems encountered are also appearing in Python 3.5 betas, it is now a "next Python released(es)" tracking bug.

Okay after I compiled 3.5.0rc1 yesterday I ran it once and apart from T106209 I got three failures:

======================================================================
ERROR: test_add_claim_with_qualifier (tests.wikibase_edit_tests.TestWikibaseWriteGeneral)
Test adding a claim with a qualifier to an item and a property.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/xzise/Programms/pywikibot/core/tests/wikibase_edit_tests.py", line 83, in test_add_claim_with_qualifier
    item.claims['P115'][0].addQualifier(end_date)
KeyError: 'P115'

======================================================================
ERROR: test_create_topic (tests.flow_edit_tests.TestFlowCreateTopic)
Test creation of topic.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/xzise/Programms/pywikibot/core/tests/flow_edit_tests.py", line 37, in test_create_topic
    first_post = topic.replies()[0]
  File "/home/xzise/Programms/pywikibot/core/pywikibot/flow.py", line 232, in replies
    return self.root.replies(format=format, force=force)
  File "/home/xzise/Programms/pywikibot/core/pywikibot/flow.py", line 214, in root
    self._root = Post.fromJSON(self, self.uuid, self._data)
  File "/home/xzise/Programms/pywikibot/core/pywikibot/flow.py", line 309, in fromJSON
    post = cls(page, post_uuid)
  File "/home/xzise/Programms/pywikibot/core/pywikibot/flow.py", line 284, in __init__
    raise NoPage(page, 'Topic must exist: %s')
pywikibot.exceptions.NoPage: Topic must exist: [[test:Topic:Sn9e87gom98hr7sd]]

======================================================================
FAIL: test_label_remove (tests.wikibase_edit_tests.TestWikibaseWriteGeneral)
Test adding a Farsi and English label and removing the Farsi one.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/xzise/Programms/pywikibot/core/tests/wikibase_edit_tests.py", line 56, in test_label_remove
    self.assertNotIn('fa', item.labels.keys())
AssertionError: 'fa' unexpectedly found in dict_keys(['en', 'fa'])

All these look like race conditions as they edit something almost directly before the test and then test that something. For example the last one was pretty reproducible but as soon as I waited some time between the two requests I couldn't get failures. I also experimented with it more and I got also a failure sooner into that test where the addition of a label wasn't noticed. Also after T106212's problem with dictionaries I checked the text returned by the API in the last failure and that already did contain both entries. So it's not like the label dict screwed up.

Not sure if Python 3.5 has made requests faster and so that it now doesn't give the server enough time to update itself and the server is returning outdated stuff.

Change 242409 had a related patch set uploaded (by XZise):
[FIX] Don't stop iterators using StopIteration

https://gerrit.wikimedia.org/r/242409

In T114464: Pywikibot 2.0 should be Python 2.7, 3.3 and 3.4 only I suggest that 2.0 should not extend to supporting 3.5+. Would be good to get opinions on that.

XZise added a comment.Oct 2 2015, 12:18 PM

Afaik it supports 3.5 even without fixing getargspec. It'll just report deprecation warnings (which are by default disabled).

Change 242409 merged by jenkins-bot:
[FIX] Don't stop iterators using StopIteration

https://gerrit.wikimedia.org/r/242409

Dalba added a subscriber: Dalba.Jul 21 2016, 4:06 PM
Dalba closed this task as Resolved.May 21 2018, 1:07 PM
Dalba claimed this task.

Right now we have tests running on python 3.5 and 3.6 on both travis-ci and appveyor.