Upgrade tox on CI instances
Closed, ResolvedPublic

Description

The current version of tox available on the CI instances is pretty old 1.9.2 and was pinned a while ago in T46443.
It would be possible to upgrade tox to a newer version?

Two natural candidates for the version are:

  • 2.5.0 available in jessie-backports
  • 2.7.0 latest on PyPi
Volans created this task.Jul 4 2017, 8:11 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 4 2017, 8:11 AM

It is probably best to stick to the Debian packages. While at it we should do the same for pip and setuptools. They are all installed from pypi since the versions available in Trusty were obsolete. Might still not be recent enough for Jessie though.

modules/contint/manifests/packages/python.pp
# Bring in fresh pip. The Trusty package does not provide wheels cache
# https://pip.pypa.io/en/latest/news.html
package { 'pip':
    ensure   => '8.1.2',
    provider => 'pip',
    require  => Package['python-pip'],  # eggs and chicken
}   

# Bring tox/virtualenv... from pip  T46443
package { 'tox':
    ensure   => '1.9.2',
    provider => 'pip',
    require  => Package['pip'],  # Fresh pip version
}
# 'pip install pip' deletes /usr/bin/pip :(
file { '/usr/bin/pip':
    ensure  => link,
    target  => '/usr/local/bin/pip',
    owner   => 'root',
    group   => 'root',
    require => Package['pip'],
}
package { 'setuptools':
    ensure   => present,
    provider => 'pip',
    require  => Package['python-pip'],
}

Once changed, there are all those jobs that needs to be checked:

apps-android-wikipedia-tox-jessie
integration-config-qa
integration-config-tox-dib-jessie
integration-config-tox-jessie
integration-jjb-config-diff
integration-zuul-layoutdiff
labs-tox-jessie
mw-tools-releng-tox-doc-publish
mw-tools-scap-tox-doc-publish
mwgate-tox-jessie
operations-puppet-cache-update-jessie
operations-puppet-tests-jessie
pywikibot-core-tox-doc-jessie
pywikibot-core-tox-doc-publish
pywikibot-core-tox-jessie
pywikibot-core-tox-nose-jessie
pywikibot-core-tox-nose34-jessie
pywikibot-tests-beta-cluster
pywikibot-wikibase-tox-nose-jessie
pywikibot-wikibase-tox-nose34-jessie
tox-doc-jessie
tox-jessie
tox-py27-coverage-publish

Most of them are running the same commands, but it is still worth checking :-]

Change 368616 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] contint: upgrade tox

https://gerrit.wikimedia.org/r/368616

Change 369604 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Pass PY_COLORS for tox 2.0.0

https://gerrit.wikimedia.org/r/369604

Change 369604 merged by jenkins-bot:
[integration/config@master] Pass PY_COLORS for tox 2.0.0

https://gerrit.wikimedia.org/r/369604

Change 368616 merged by Volans:
[operations/puppet@production] contint: upgrade tox

https://gerrit.wikimedia.org/r/368616

Mentioned in SAL (#wikimedia-releng) [2017-08-02T08:32:02Z] <hashar> Regenerating Nodepool jessie image to upgrade tox from 1.9.2 to 2.5.0 - T169602

Mentioned in SAL (#wikimedia-releng) [2017-08-02T08:44:05Z] <hashar> Image snapshot-ci-jessie-1501662758 in wmflabs-eqiad is ready - T169602

Mentioned in SAL (#wikimedia-releng) [2017-08-02T09:02:18Z] <hashar> Regenerating Nodepool Jessie image from scratch to get rid of tox 1.9.2 installed under /usr/local - T169602

Change 369620 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] contint: Package[tox] -> Package[python-tox]

https://gerrit.wikimedia.org/r/369620

Change 369620 merged by Volans:
[operations/puppet@production] contint: Package[tox] -> Package[python-tox]

https://gerrit.wikimedia.org/r/369620

Mentioned in SAL (#wikimedia-releng) [2017-08-02T10:51:05Z] <hashar> Image snapshot-ci-jessie-1501670727 in wmflabs-eqiad is ready - T169602

WARNING: apparently tox now defaults to use python3 which breaks a bunch of jobs :(

Change 369632 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] contint: revert tox installation from Debian package

https://gerrit.wikimedia.org/r/369632

Change 369634 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] contint: upgrade tox to 2.5.0 using pip

https://gerrit.wikimedia.org/r/369634

Change 369632 merged by Marostegui:
[operations/puppet@production] contint: revert tox installation from Debian package

https://gerrit.wikimedia.org/r/369632

Change 369634 merged by Marostegui:
[operations/puppet@production] contint: upgrade tox to 2.5.0 using pip

https://gerrit.wikimedia.org/r/369634

Mentioned in SAL (#wikimedia-releng) [2017-08-02T11:31:55Z] <hashar> Image snapshot-ci-jessie-1501673225 in wmflabs-eqiad is ready T169602

hashar closed this task as Resolved.Aug 2 2017, 11:39 AM
hashar claimed this task.

The tox based jobs now use version 2.5.0 installed from pypi.