Page MenuHomePhabricator

CI: run tests with multiple Python3 versions
Closed, ResolvedPublic

Description

Regarding Python 3 currently our CI allows only to run tests with Python 3.4, that is less than a year from end of life (2019/03). In production we're quickly moving towards Debian Stretch that ships 3.5 and is most likely that newer developed software might not be backward compatible with Python 3.4. I've already a repo in which this is true.

Given that is possible to have multiple Python versions installed on a host, I think that our CI should have multiple Python versions installed allowing tox to run tests on multiple Python versions as defined in the tox.ini file. I'd say 2.7, 3.4, 3.5 and 3.6 should all be available.

Details

Related Gerrit Patches:

Event Timeline

Volans created this task.Apr 9 2018, 8:18 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 9 2018, 8:18 AM

Yup that is definitely needed, but I am not sure how to implement it.

For the context: the CI tox jobs are all based on a Jessie container and get python 2.7.9 and 3.4 provided by the official Debian package. The containers are defined in integration/config (eg: dockerfiles/tox/Dockerfile.template) and build with docker-pkg.

Most probably we should switch to Stretch (python 2.7.13 and 3.5.3) and add additional pythons (3.4, 3.6) in some way. pyenv is most probably the defacto standard for that. I have no idea how tox integrates with it though. Ideally we would have the container shipping several pythons and get tox to find them somehow. Or to say otherwise, given:

tox.ini
[tox]
skipsdist = True  # I have no setup.py
envlist = py27, py34, py35, py36

[testenv]
cmd = python --version

Running tox -q would yield:

Python 2.7.13
Python 3.4.8
Python 3.5.3
Python 3.6.3

I have no idea how pyenv works though. Most probably it has some magic to setup everything in a way that is transparent for tox (which afaik looks for python3.4 python3.5 etc executables in PATH).

D3r1ck01 added a subscriber: D3r1ck01.

Can someone please look into pyenv and goes through its documentation to find out how we can use it? If we can manage to have it to provide /usr/bin/python3.4 /usr/bin/python3.5 etc, then tox would magically discover the version and that will solve the task. I can't allocate time to explore pyenv though and would thus need some help on that front.

Once pyenv configuration/integration is sorted out, it can be added to the CI Docker containers releng/tox*

ArielGlenn added a comment.EditedDec 12 2018, 9:33 AM

I note that there do not seem to be debian packages for pyenv.

https://github.com/pyenv/pyenv/issues/138

It also seems that in order to use pyenv to switch between python versions, pyenv should be used to build these versions from source; see https://github.com/pyenv/pyenv/issues/1244 We may be able to work around it, though this is unclear. We would have to go through the python patches in the pyenv python-build directory for the versions we want and see how necessary they are.

faidon added a subscriber: faidon.Dec 17 2018, 11:03 AM

This was quite complicated but I've managed to forward-port 3.4 and backpored 3.6 and 3.7 to stretch. These are now included in the component component/pyall of suite stretch-wikimedia and they can be installed like one would normally install python (apt install python3.{4,5,6,7}-venv should do it).

They won't be useful for all use cases, but they should be for building venvs, e.g. with tox, and thus can and probably should be used for CI. I've tested these on my local stretch system by running tox against cumin and keyholder and they work fine.

Change 480428 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[integration/config@master] docker: Support multiple Python 3 versions in tox image

https://gerrit.wikimedia.org/r/480428

Change 480428 merged by jenkins-bot:
[integration/config@master] docker: Support multiple Python 3 versions in tox image

https://gerrit.wikimedia.org/r/480428

Change 480570 had a related patch set uploaded (by Volans; owner: Volans):
[integration/config@master] tox-docker: bump default timeout to 10 minutes

https://gerrit.wikimedia.org/r/480570

Change 480570 merged by jenkins-bot:
[integration/config@master] tox-docker: bump default timeout to 10 minutes

https://gerrit.wikimedia.org/r/480570

hashar closed this task as Resolved.Dec 18 2018, 7:43 PM
hashar assigned this task to faidon.
hashar added a subscriber: Legoktm.

@faidon that looks easier than pyenv in the end. Did you need any specific patches for the ports? If so we might want to have them in Gerrit.

@Legoktm looks like the CI container works fine. One of the repo reached the default 3 minutes timeout, got bumped to 10 minutes.

Thank you!

Change 484200 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: fix tox-pyspark gpg under stretch

https://gerrit.wikimedia.org/r/484200

Change 484200 merged by Hashar:
[integration/config@master] docker: fix tox-pyspark gpg under stretch

https://gerrit.wikimedia.org/r/484200

Change 484202 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: tox-pyspark Java 7 -> 8

https://gerrit.wikimedia.org/r/484202

Change 484202 merged by Hashar:
[integration/config@master] docker: tox-pyspark Java 7 -> 8

https://gerrit.wikimedia.org/r/484202

Change 484207 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Update pywikibot jobs to releng/tox-pywikibot:0.3.0

https://gerrit.wikimedia.org/r/484207

Change 484209 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Update search/mjolnir tox job to 0.3.0 container

https://gerrit.wikimedia.org/r/484209

Change 484207 merged by jenkins-bot:
[integration/config@master] Update pywikibot jobs to releng/tox-pywikibot:0.3.0

https://gerrit.wikimedia.org/r/484207

Change 484209 merged by jenkins-bot:
[integration/config@master] Update search/mjolnir tox job to 0.3.0 container

https://gerrit.wikimedia.org/r/484209

Mentioned in SAL (#wikimedia-releng) [2019-01-14T14:31:23Z] <hashar> Nuked Castor cache for all *tox* jobs. Some might have cached binary wheels compiled against a lib that is no more existing (eg libmysqlclient.so.18 for mysql-python). Follow up the jessie -> stretch upgrade # T191764