Page MenuHomePhabricator
Create Task
Maniphest T130463

Install issues on older (Jessie 8.1) nodes
Closed, ResolvedPublic
Actions

Description

Mar 19 16:57:56 ores-worker-01 celery[31510]: return imp(module, package=package)
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/venv/lib/python3.4/importlib/__init__.py", line 109, in import_module
Mar 19 16:57:56 ores-worker-01 celery[31510]: return _bootstrap._gcd_import(name[level:], package, level)
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "<frozen importlib._bootstrap>", line 2254, in _gcd_import
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "<frozen importlib._bootstrap>", line 1200, in _load_unlocked
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "<frozen importlib._bootstrap>", line 1129, in _exec
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "<frozen importlib._bootstrap>", line 1471, in exec_module
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "<frozen importlib._bootstrap>", line 321, in _call_with_frames_removed
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/config/ores_celery.py", line 8, in <module>
Mar 19 16:57:56 ores-worker-01 celery[31510]: from ores.score_processors import Celery
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/venv/lib/python3.4/site-packages/ores/score_processors/__init__.py", line 3, in <module>
Mar 19 16:57:56 ores-worker-01 celery[31510]: from .celery import Celery
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/venv/lib/python3.4/site-packages/ores/score_processors/celery.py", line 7, in <module>
Mar 19 16:57:56 ores-worker-01 celery[31510]: import revscoring.errors
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/venv/lib/python3.4/site-packages/revscoring/__init__.py", line 117, in <module>
Mar 19 16:57:56 ores-worker-01 celery[31510]: from .scorer_models import ScorerModel
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/venv/lib/python3.4/site-packages/revscoring/scorer_models/__init__.py", line 34, in <module>
Mar 19 16:57:56 ores-worker-01 celery[31510]: from .svc import SVC, LinearSVC, RBFSVC
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/venv/lib/python3.4/site-packages/revscoring/scorer_models/svc.py", line 21, in <module>
Mar 19 16:57:56 ores-worker-01 celery[31510]: from sklearn import svm
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/venv/lib/python3.4/site-packages/sklearn/svm/__init__.py", line 13, in <module>
Mar 19 16:57:56 ores-worker-01 celery[31510]: from .classes import SVC, NuSVC, SVR, NuSVR, OneClassSVM, LinearSVC, \
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/venv/lib/python3.4/site-packages/sklearn/svm/classes.py", line 4, in <module>
Mar 19 16:57:56 ores-worker-01 celery[31510]: from .base import _fit_liblinear, BaseSVC, BaseLibSVM
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/venv/lib/python3.4/site-packages/sklearn/svm/base.py", line 12, in <module>
Mar 19 16:57:56 ores-worker-01 celery[31510]: from ..multiclass import _ovr_decision_function
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/venv/lib/python3.4/site-packages/sklearn/multiclass.py", line 44, in <module>
Mar 19 16:57:56 ores-worker-01 celery[31510]: from .metrics.pairwise import euclidean_distances
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/venv/lib/python3.4/site-packages/sklearn/metrics/__init__.py", line 33, in <module>
Mar 19 16:57:56 ores-worker-01 celery[31510]: from . import cluster
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/venv/lib/python3.4/site-packages/sklearn/metrics/cluster/__init__.py", line 21, in <module>
Mar 19 16:57:56 ores-worker-01 celery[31510]: from .bicluster import consensus_score
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/venv/lib/python3.4/site-packages/sklearn/metrics/cluster/bicluster/__init__.py", line 1, in <module>
Mar 19 16:57:56 ores-worker-01 celery[31510]: from .bicluster_metrics import consensus_score
Mar 19 16:57:56 ores-worker-01 celery[31510]: File "/srv/ores/venv/lib/python3.4/site-packages/sklearn/metrics/cluster/bicluster/bicluster_metrics.py", line 6, in <module>
Mar 19 16:57:56 ores-worker-01 celery[31510]: from sklearn.utils.validation import check_arrays
Mar 19 16:57:56 ores-worker-01 celery[31510]: ImportError: cannot import name 'check_arrays'

We get this error when starting up services on debian jessie 8.1 nodes. It looks like maybe our wheels (build on Debian 8.3) are not compatible.

Event Timeline

Halfak created this task.Mar 19 2016, 5:00 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 19 2016, 5:00 PM
Ladsgroup subscribed.Mar 19 2016, 5:02 PM

mostly it seems sci-kit learn isn't compatible. I check more carefully

Ladsgroup added a comment.Mar 19 2016, 6:27 PM

OK. After lots of checks, we realized we should not install wheels when a virtualenv has some stuff installed in it already, probably causing conflicts in dependencies. So every time, we should tear down the virtualenv and build a new one. @Halfak is adding that part to the fabfile (and I'll do it with the scap checks later)

Halfak added a comment.Mar 19 2016, 6:28 PM

See https://github.com/pypa/pip/issues/214

Ladsgroup moved this task from Parked to Completed on the Machine-Learning-Team (Active Tasks) board.Mar 20 2016, 11:43 PM
Halfak closed this task as Resolved.Mar 21 2016, 3:54 PM
Halfak claimed this task.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL