Page MenuHomePhabricator
Create Task
Maniphest T267803

python installation in `mediawiki-services-wikispeech-mishkal` break the system-level python3
Open, Needs TriagePublic
Actions

Description

We have current failures while running debmonitor because it's impossible to install any python 3 package in that image:

etting up python3-minimal (3.7.3-1) ...
Traceback (most recent call last):
  File "/usr/bin/py3compile", line 25, in <module>
    import logging
  File "/usr/lib/python3.7/logging/__init__.py", line 26, in <module>
    import sys, os, time, io, traceback, warnings, weakref, collections.abc
  File "/usr/lib/python3.7/traceback.py", line 3, in <module>
    import collections
  File "/usr/lib/python3.7/collections/__init__.py", line 27, in <module>
    from reprlib import recursive_repr as _recursive_repr
  File "/opt/lib/python/site-packages/reprlib/__init__.py", line 7, in <module>
    raise ImportError('This package should not be accessible on Python 3. '
ImportError: This package should not be accessible on Python 3. Either you are trying to run from the python-future src folder or your installation of python-future is corrupted.
dpkg: error processing package python3-minimal (--configure):
 installed python3-minimal package post-installation script subprocess returned error exit status 1

Event Timeline

Joe created this task.Nov 12 2020, 3:08 PM
Restricted Application added a project: Wikispeech-Jobrunner. · View Herald TranscriptNov 12 2020, 3:08 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Lokal_Profil moved this task from Unsorted to Synthesis on the Wikispeech-Text-to-Speech board.EditedNov 26 2020, 10:57 AM
Lokal_Profil added projects: User-kalle, Wikispeech-WMSE.
Lokal_Profil added subscribers: kalle, Lokal_Profil.

Thanks for flagging this!

@kalle can you take a look at this possibly rope in STTS if needed

Lokal_Profil added a comment.Nov 26 2020, 11:01 AM
This comment was removed by Lokal_Profil.
kalle added a comment.Nov 26 2020, 12:51 PM

@Joe Can you please help me understand how we can we reproduce this error locally, how and when it occurs?

None of these cause an error for me locally:

blubber .pipeline/blubber.yaml test | docker build --tag wikispeech-mishkal-test --file - .
blubber .pipeline/blubber.yaml production | docker build --tag wikispeech-mishkal --file - .

This is the output from the later:

Sending build context to Docker daemon    177MB
Step 1/19 : FROM docker-registry.wikimedia.org/wikimedia-buster
 ---> 5877a8565b78
Step 2/19 : USER "root"
 ---> Using cache
 ---> a95fa9052bfe
Step 3/19 : ENV HOME="/root"
 ---> Using cache
 ---> 42fda0f4e24e
Step 4/19 : ENV DEBIAN_FRONTEND="noninteractive"
 ---> Using cache
 ---> 6f72b44374d7
Step 5/19 : RUN apt-get update && apt-get install -y "ca-certificates" "python-setuptools" "wget" && rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> cbe096cf14fc
Step 6/19 : RUN python2.7 "-m" "easy_install" "pip" && python2.7 "-m" "pip" "install" "-U" "setuptools" "wheel" "tox"
 ---> Using cache
 ---> af63d0984914
Step 7/19 : RUN groupadd -o -g "65533" -r "somebody" && useradd -o -m -d "/home/somebody" -r -g "somebody" -u "65533" "somebody" && mkdir -p "/srv/mishkal" && chown "65533":"65533" "/srv/mishkal" && mkdir -p "/opt/lib" && chown "65533":"65533" "/opt/lib"
 ---> Using cache
 ---> 28825c8d125c
Step 8/19 : RUN groupadd -o -g "900" -r "runuser" && useradd -o -m -d "/home/runuser" -r -g "runuser" -u "900" "runuser"
 ---> Using cache
 ---> c2a3a20dc017
Step 9/19 : USER "somebody"
 ---> Using cache
 ---> 00da1b45a126
Step 10/19 : ENV HOME="/home/somebody"
 ---> Using cache
 ---> 3648503382fe
Step 11/19 : WORKDIR "/srv/mishkal"
 ---> Using cache
 ---> a805f7e22e8a
Step 12/19 : ENV PIP_FIND_LINKS="file:///opt/lib/python" PIP_WHEEL_DIR="/opt/lib/python"
 ---> Using cache
 ---> d3b41fe91144
Step 13/19 : RUN mkdir -p "/opt/lib/python"
 ---> Using cache
 ---> 9c6b2503cdc5
Step 14/19 : COPY --chown=65533:65533 ["requirements.txt", "./"]
 ---> a84d47d0bec0
Step 15/19 : RUN python2.7 "-m" "pip" "wheel" "-r" "requirements.txt" && python2.7 "-m" "pip" "install" "--target" "/opt/lib/python/site-packages" "-r" "requirements.txt"
 ---> Running in 5b0fcb46c176
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Looking in links: file:///opt/lib/python
Collecting pyarabic>=0.6.2
  Downloading PyArabic-0.6.10.tar.gz (108 kB)
Collecting CodernityDB==0.4.2
  Downloading CodernityDB-0.4.2.tar.gz (45 kB)
Collecting libqutrub>=1.0
  Downloading libqutrub-1.2.4.1-py2-none-any.whl (136 kB)
  Saved /opt/lib/python/libqutrub-1.2.4.1-py2-none-any.whl
Collecting naftawayh>=0.2
  Downloading Naftawayh-0.4-py2-none-any.whl (329 kB)
  Saved /opt/lib/python/Naftawayh-0.4-py2-none-any.whl
Collecting repr==0.3.1
  Downloading repr-0.3.1-py2.py3-none-any.whl (7.8 kB)
  Saved /opt/lib/python/repr-0.3.1-py2.py3-none-any.whl
Collecting tashaphyne==0.3.1
  Downloading Tashaphyne-0.3.1-py2-none-any.whl (19 kB)
  Saved /opt/lib/python/Tashaphyne-0.3.1-py2-none-any.whl
Collecting arramooz-pysqlite==0.1
  Downloading arramooz_pysqlite-0.1-py2-none-any.whl (7.0 MB)
  Saved /opt/lib/python/arramooz_pysqlite-0.1-py2-none-any.whl
Collecting qalsadi<0.3.5,>=0.2
  Downloading qalsadi-0.3.4-py2-none-any.whl (246 kB)
  Saved /opt/lib/python/qalsadi-0.3.4-py2-none-any.whl
Collecting mysam-tagmanager==0.1
  Downloading mysam_tagmanager-0.1-py2-none-any.whl (17 kB)
  Saved /opt/lib/python/mysam_tagmanager-0.1-py2-none-any.whl
Collecting alyahmor==0.1
  Downloading alyahmor-0.1-py2-none-any.whl (37 kB)
  Saved /opt/lib/python/alyahmor-0.1-py2-none-any.whl
Collecting future==0.18.2
  Downloading future-0.18.2.tar.gz (829 kB)
Collecting six
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
  Saved /opt/lib/python/six-1.15.0-py2.py3-none-any.whl
Skipping libqutrub, due to already being wheel.
Skipping naftawayh, due to already being wheel.
Skipping repr, due to already being wheel.
Skipping tashaphyne, due to already being wheel.
Skipping arramooz-pysqlite, due to already being wheel.
Skipping qalsadi, due to already being wheel.
Skipping mysam-tagmanager, due to already being wheel.
Skipping alyahmor, due to already being wheel.
Skipping six, due to already being wheel.
Building wheels for collected packages: pyarabic, CodernityDB, future
  Building wheel for pyarabic (setup.py): started
  Building wheel for pyarabic (setup.py): finished with status 'done'
  Created wheel for pyarabic: filename=PyArabic-0.6.10-py2-none-any.whl size=113321 sha256=ff8b0a7f582fa29b652c955690db7a13af6685452b1906f20549161c4de2b2e3
  Stored in directory: /home/somebody/.cache/pip/wheels/d5/1f/52/d94ca1cd173aa386178bee794c8554a51e2add9871d13e7578
  Building wheel for CodernityDB (setup.py): started
  Building wheel for CodernityDB (setup.py): finished with status 'done'
  Created wheel for CodernityDB: filename=CodernityDB-0.4.2-py2-none-any.whl size=58841 sha256=1ce56ab072e7ecf474d32257cb506e239adf7f5ede28d4f397838f63443b7e06
  Stored in directory: /home/somebody/.cache/pip/wheels/c7/fb/5d/a73071be87cff98711777da3f514b810bf99cdc82925825422
  Building wheel for future (setup.py): started
  Building wheel for future (setup.py): finished with status 'done'
  Created wheel for future: filename=future-0.18.2-py2-none-any.whl size=502566 sha256=9b8cfccdcc8e2040fbe3aa02d161736de7cc8d90f70287e52b23b31273cb5ee0
  Stored in directory: /home/somebody/.cache/pip/wheels/5f/11/0c/aad680baf5ef4fbcbab992c9f03e1130357e0c173a4fdabfff
Successfully built pyarabic CodernityDB future
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Looking in links: file:///opt/lib/python
Processing /opt/lib/python/PyArabic-0.6.10-py2-none-any.whl
Processing /opt/lib/python/CodernityDB-0.4.2-py2-none-any.whl
Processing /opt/lib/python/libqutrub-1.2.4.1-py2-none-any.whl
Processing /opt/lib/python/Naftawayh-0.4-py2-none-any.whl
Processing /opt/lib/python/repr-0.3.1-py2.py3-none-any.whl
Processing /opt/lib/python/Tashaphyne-0.3.1-py2-none-any.whl
Processing /opt/lib/python/arramooz_pysqlite-0.1-py2-none-any.whl
Processing /opt/lib/python/qalsadi-0.3.4-py2-none-any.whl
Processing /opt/lib/python/mysam_tagmanager-0.1-py2-none-any.whl
Processing /opt/lib/python/alyahmor-0.1-py2-none-any.whl
Processing /opt/lib/python/future-0.18.2-py2-none-any.whl
Processing /opt/lib/python/six-1.15.0-py2.py3-none-any.whl
Installing collected packages: pyarabic, CodernityDB, libqutrub, tashaphyne, naftawayh, six, repr, arramooz-pysqlite, qalsadi, mysam-tagmanager, alyahmor, future
Successfully installed CodernityDB-0.4.2 alyahmor-0.1 arramooz-pysqlite-0.1 future-0.18.2 libqutrub-1.2.4.1 mysam-tagmanager-0.1 naftawayh-0.4 pyarabic-0.6.10 qalsadi-0.3.4 repr-0.3.1 six-1.15.0 tashaphyne-0.3.1
Removing intermediate container 5b0fcb46c176
 ---> 296a6ec4491a
Step 16/19 : COPY --chown=65533:65533 [".", "."]
 ---> 3b0c27e8329a
Step 17/19 : ENV PATH="/opt/lib/python/site-packages/bin:${PATH}" PIP_NO_INDEX="1" PYTHONPATH="/opt/lib/python/site-packages"
 ---> Running in ed82e3bd079d
Removing intermediate container ed82e3bd079d
 ---> 98cd10d26805
Step 18/19 : ENTRYPOINT ["./blubber-entrypoint-test.sh"]
 ---> Running in cd08fe0620ae
Removing intermediate container cd08fe0620ae
 ---> 4c96993d5c2c
Step 19/19 : LABEL blubber.variant="test" blubber.version="0.8.0+ba236df"
 ---> Running in d6de4ada2392
Removing intermediate container d6de4ada2392
 ---> d3076f47a299
Successfully built d3076f47a299
Successfully tagged wikispeech-mishkal-test:latest
Sending build context to Docker daemon    177MB
kalle added a comment.Nov 30 2020, 12:18 PM

Or perhaps the debmon error is something that occurs when you inject it in the image we build? Or something along that line? If so, please let me know how I can reproduce that behavior in order to help figuring this out.

Lokal_Profil moved this task from 🥴 Backlog to 🤕 Watching on the User-kalle board.Dec 1 2020, 9:53 AM
kalle added a comment.Dec 1 2020, 9:53 AM

Btw, this installation of Mishkal is running on Py2.7. There is a new version that runs on Py3.8, but the WMF debian base image only support Py3.7.

Lokal_Profil moved this task from Incoming to Backlog on the Wikispeech-Jobrunner board.Dec 3 2020, 9:16 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL