Page MenuHomePhabricator

Request to install exiv on Toolforge
Closed, ResolvedPublic

Description

I'd like to use py3exiv2 for a bot task.

Judging by http://py3exiv2.tuxfamily.org/downloads, it seems that these commands need to be run:

sudo add-apt-repository ppa:vincent-vandevyvre/vvv
sudo apt-get update
sudo apt-get install python3-exiv2

The bot task will use exiv to transfer metadata from one image to the other.

Related Objects

Event Timeline

Unfortunately, PPA's cannot be used due to security concerns, but I think the following should work:

  • apt-get install libexiv2-dev to install the compilation dependency,
  • pip install python3-exiv2 in a virtualenv to get the python library. That would not require sudo as long as the exiv2-dev dependency is installed.

We have python2 support for this on the Stretch grid:

$ dpkg-installed |grep exiv
libexiv2-14:amd64    0.25-3.1+deb9u1
python-pyexiv2       0.3.2-8+b3

Debian does not seem to package this library for python3, so as @valhallasw mentioned above the best we can do here is to add the libexiv2-dev package so that you can install https://pypi.org/project/py3exiv2/ into a virtual environment.

I fiddled around a bit with docker to try and find the minimal dependencies needed to pip install py3exiv2:

$ docker run -it python:3-stretch bash
[...]

# pip install py3exiv2
[...]
src/exiv2wrapper.hpp:32:27: fatal error: exiv2/image.hpp: No such file or directory

# apt-get update
[...]

# apt-get install libexiv2-dev
[...]

# pip install py3exiv2
[...]
src/exiv2wrapper.hpp:35:28: fatal error: boost/python.hpp: No such file or directory

# apt-get install libboost-python-dev
[...]

# pip install py3exiv2
[...]
Installing collected packages: py3exiv2
Successfully installed py3exiv2-0.4.0

so libboost-python-dev should also be available. Looking at the puppet manifests, I think this may be the case for the ubuntu dev hosts, but not for the new stretch hosts.

Change 486350 had a related patch set uploaded (by Bstorm; owner: Bstorm):
[operations/puppet@production] toolforge: change dev_environ into shell_environ

https://gerrit.wikimedia.org/r/486350

Change 486350 merged by Bstorm:
[operations/puppet@production] toolforge: change dev_environ into shell_environ

https://gerrit.wikimedia.org/r/486350

Change 486401 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/puppet@production] toolforge: install libboost-python-dev on grid nodes

https://gerrit.wikimedia.org/r/486401

Change 486401 merged by Bstorm:
[operations/puppet@production] toolforge: install libboost-python-dev on grid nodes

https://gerrit.wikimedia.org/r/486401

Change 486413 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/puppet@production] toolforge: install libexiv2-dev on grid nodes

https://gerrit.wikimedia.org/r/486413

Change 486413 merged by Bstorm:
[operations/puppet@production] toolforge: install libexiv2-dev on grid nodes

https://gerrit.wikimedia.org/r/486413

bd808 claimed this task.
$ python3 -mvenv  venv-bd808-T213965
$ ./venv-bd808-T213965/bin/pip install py3exiv2
Collecting py3exiv2
  Using cached https://files.pythonhosted.org/packages/95/e3/d43f8635d85074f1e128ca42b7abda7655eb10d592081625f6125b72d356/py3exiv2-0.4.0.tar.gz
Building wheels for collected packages: py3exiv2
  Running setup.py bdist_wheel for py3exiv2 ... error
  Complete output from command /tmp/venv-bd808-T213965/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-3nljoxfe/py3exiv2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpe2ztqm1_pip-wheel- --python-tag cp35:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

  ----------------------------------------
  Failed building wheel for py3exiv2
  Running setup.py clean for py3exiv2
Failed to build py3exiv2
Installing collected packages: py3exiv2
  Running setup.py install for py3exiv2 ... done
Successfully installed py3exiv2-0.4.0
$ ./venv-bd808-T213965/bin/pip freeze
pkg-resources==0.0.0
py3exiv2==0.4.0
$ ./venv-bd808-T213965/bin/python3
Python 3.5.3 (default, Sep 27 2018, 17:25:39)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyexiv2
>>> help(pyexiv2)
Help on package pyexiv2:

NAME
    pyexiv2 - Manipulation of EXIF, IPTC and XMP metadata and thumbnails embedded in images.
...