Page MenuHomePhabricator

tox v4 recreate test environment sharing the same envdir / env_dir
Open, Needs TriagePublic

Description

There are a few repositories sharing the same environment dir to avoid recreating an environment when it shares the same python version and set of dependencies as another environment but differs in the command being run. One such use case is in integration/quibble.git which has cover and integration env which are the same as a py3-unit env albeit with a different command. Here is a summary of the tox.ini:

tox.ini
envdir =
# tox v3 provisionining would fail with:
# ConfigError:envdir must not equal toxinidir
    .tox: {toxworkdir}/.pkg
    py3-unit: {toxworkdir}/venv-py3
    py3-lint: {toxworkdir}/venv-lint-py3

    check: {toxworkdir}/venv-py3
    black: {toxworkdir}/black
    integration: {toxworkdir}/venv-py3
    cover: {toxworkdir}/venv-py3
    venv: {toxworkdir}/venv-py3

That has however never been supported in tox 3 ( https://github.com/tox-dev/tox/issues/425#issuecomment-801296002 ) and is explicitly not supported in tox v4 ( https://github.com/tox-dev/tox/issues/2788 ). The comparison is done based on the testenv name (py3-unit, cover, integration) and the type of runner (VirtualEnvRunner). Any mismatch cause a recreation, here is an example with tox v4 without any .tox:

check: install_deps> python -I -m pip install '.[test]'
...
check: install_package_deps> python -I -m pip install 'GitPython~=3.1.0' PyYAML extras requests six
check: install_package> python -I -m pip install --force-reinstall --no-deps /home/hashar/projects/integration/quibble/.tox/.tmp/package/1/quibble-1.5.6.post5+git.1fe36f58.dirty.tar.gz
check: OK ✔ in 19.41 seconds
...
py3-unit: recreate env because env type changed from {'name': 'check', 'type': 'VirtualEnvRunner'} to {'name': 'py3-unit', 'type': 'VirtualEnvRunner'}
py3-unit: remove tox env folder /home/hashar/projects/integration/quibble/.tox/venv-py3
py3-unit: install_deps> python -I -m pip install '.[test]'
py3-unit: install_package_deps> python -I -m pip install 'GitPython~=3.1.0' PyYAML extras requests six

So that even if check and py3-unit shares the same basepython, type of runner and set of dependencies, a new venv gets created since the names mismatch.

There is a plugin to filter out the test environment name from the configuration when doing the comparison: tox-ignore-env-name-mismatch. Then I guess it is easier to stop mangling with the environment directories and have one directory per environment. That causes some more disk space to be used, but speed wise the dependencies would be installed from the wheel cache so that should be rather fast.

That also let me get rid of a tox v3 back compatibility hack to provision tox v4 in a .pkg/.package env:

tox.ini
envdir =
    .tox = {toxworkdir}/.pkg

Event Timeline

Change 964418 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/quibble@master] tox: remove envdir optimizations

https://gerrit.wikimedia.org/r/964418

Change 964418 merged by jenkins-bot:

[integration/quibble@master] tox: remove envdir optimizations

https://gerrit.wikimedia.org/r/964418

thcipriani subscribed.

Plan is to document this for folks migrating from toxv3→toxv4

Change 966847 had a related patch set uploaded (by Hashar; author: Hashar):

[operations/software/puppet-compiler@master] tox: remove envdir optimizations

https://gerrit.wikimedia.org/r/966847

Change 967150 had a related patch set uploaded (by Jbond; author: Hashar):

[operations/software/puppet-compiler@2.x] tox: remove envdir optimizations

https://gerrit.wikimedia.org/r/967150

Change 967150 merged by jenkins-bot:

[operations/software/puppet-compiler@2.x] tox: remove envdir optimizations

https://gerrit.wikimedia.org/r/967150

Change 966847 abandoned by Jbond:

[operations/software/puppet-compiler@master] tox: remove envdir optimizations

Reason:

see chain at https://gerrit.wikimedia.org/r/c/operations/software/puppet-compiler/+/967157/3

https://gerrit.wikimedia.org/r/966847

Change 982641 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/quibble@master] release: Quibble 1.6.0

https://gerrit.wikimedia.org/r/982641

Change 982641 merged by jenkins-bot:

[integration/quibble@master] release: Quibble 1.6.0

https://gerrit.wikimedia.org/r/982641

Change 982814 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] dockerfiles: update Quibble to 1.6.0

https://gerrit.wikimedia.org/r/982814

Change 982814 merged by jenkins-bot:

[integration/config@master] dockerfiles: update Quibble to 1.6.0

https://gerrit.wikimedia.org/r/982814