Page MenuHomePhabricator

Install a version of Python newer than 3.5.3 in Toolforge
Open, Needs TriagePublic

Description

e.g. current version https://github.com/SuLab/WikidataIntegrator/ only support Python 3.6+ (due to sparql-slurper dependency), but Toolforge only have Python 3.5.3. So it's not possible to run WikidataIntegrator without manually compile a newer version of Python.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 21 2019, 10:31 PM
Aklapper renamed this task from Install newer version of Python in Toolforge to Install a version of Python newer than 3.5.3 in Toolforge.Aug 21 2019, 11:03 PM
Phamhi claimed this task.Mon, Sep 30, 8:09 PM

stretch doesn't have python3.6 so as per suggestion from bstorm, I'm going to skip to python3.7 since it's available on buster

Phamhi added a comment.EditedFri, Oct 11, 9:30 PM

python37/web (specifically toollabs-webservice) needs python-pykube which doesn't look like it's available in buster

The following packages have unmet dependencies:
 toollabs-webservice : Depends: python-pykube but it is not installable
E: Unable to correct problems, you have held broken packages.
The command '/bin/sh -c apt-get update     && DEBIAN_FRONTEND=noninteractive     apt-get install --yes     toollabs-webservice     uwsgi     uwsgi-plugin-python3     && apt-get clean     && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100
Traceback (most recent call last):
  File "./build.py", line 228, in <module>
    main()
  File "./build.py", line 219, in main
    args.tag,
  File "./build.py", line 109, in build_image
    subprocess.check_call(args)
  File "/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/local/bin/docker', 'build', '-t', 'docker-registry.tools.wmflabs.org/toolforge-python37-web:testing', '--no-cache', '/Users/ll68349/wikimedia/toollabs-images/python37/web']' returned non-zero exit status 100.
(buster docker instance) # apt policy python-pykube
python-pykube:
  Installed: (none)
  Candidate: (none)
  Version table:

I think similar to https://phabricator.wikimedia.org/T200660, we need to forward-port python-pykube to buster. Will talk to @aborrero for help.

Change 543124 had a related patch set uploaded (by Phamhi; owner: Hieu Pham):
[operations/docker-images/toollabs-images@master] Update all images based on buster (T230961)

https://gerrit.wikimedia.org/r/543124

I might as well extend the scope of this ticket to update all docker images based on buster.

This means the following new language versions will be available:

  • Golang 1.11
  • JDK 11
  • PHP 7.3
  • Python 3.7
  • Ruby2.5
  • (possibly more to come)

Mentioned in SAL (#wikimedia-operations) [2019-10-15T12:13:08Z] <arturo> add copy of python-pykube and python3-pykube from stretch-wikimedia to buster-wikimedia (T230961)

Phamhi added a comment.EditedTue, Oct 15, 12:28 PM

Woot woot... confirmed.. thanks arturo for unblocking me

It looks like toollabs-webservice is not in buster-tools... will ask for help from arturo again to port this package