| Htriedman | |
| May 22 2023, 5:36 PM |
| F37751322: image.png | |
| Sep 22 2023, 4:57 PM |
| F37751313: image.png | |
| Sep 22 2023, 4:57 PM |
| F37750120: image.png | |
| Sep 22 2023, 10:47 AM |
| F37750113: image.png | |
| Sep 22 2023, 10:47 AM |
| F37747331: image.png | |
| Sep 21 2023, 4:44 PM |
| F37747328: image.png | |
| Sep 21 2023, 4:37 PM |
I'm hoping that we can enable the use of libmamba by default in conda environments, particularly for CI tasks in Gitlab.
In my experience, it's reduced 45-50 minute environment resolution processes to 4-5 minutes. I don't know exactly what is required to make this a default, but it would certainly reduce the amount of time waiting for tests to pass.
| Title | Reference | Author | Source Branch | Dest Branch | |
|---|---|---|---|---|---|
| Fix problems with the libmamaba solver | repos/data-engineering/conda-analytics!35 | btullis | fix_conda_libmamba | main | |
| Downgrade the libmamba package so that it matches conda | repos/data-engineering/conda-analytics!34 | btullis | matching-conda_solver_versions | main | |
| Add conda-libmamba-solver to the conda-analytics environment | repos/data-engineering/conda-analytics!33 | btullis | install_libmamba_solver | main | |
| Pin the version of sqlachemy used in conda-analytics | repos/data-engineering/conda-analytics!32 | btullis | pin_sqlalchemy | main | |
| Update miniconda and enable libmamba solver | repos/data-engineering/conda-analytics!28 | btullis | enable_libmamba | main |
btullis opened https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/28
Draft: Update miniconda and enable libmamba solver
Since I was already working on builds of conda-analytics, I've made a draft MR to try this out.
https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/28
No guarantee of success at the moment, but maybe something to build on.
Thank you for working on this, @BTullis.
I would love to see this completed! For me, the Mamba solver has not only sped up installations by at least 1000%, it also solved a problem for both @JAnstee_WMF and myself which blocked us for installing any new packages in our Conda environments.
(The details: we both installed packages using Conda and, as a side effect, had Conda update itself from ~4.13 to ~23.1. After that, every attempt to install or update a packaged failed with the error "ResolvePackageNotFound: conda=4.13.0" (the old version!). In both our cases, creating a new environment and switching to the Mamba solver before installing anything else prevented that from happening.)
btullis merged https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/28
Update miniconda and enable libmamba solver
I have finished building conda-analytics version 0.0.19 and it is now on apt.wikimedia.org, ready for download.
btullis@apt1001:~$ sudo -i reprepro ls conda-analytics conda-analytics | 0.0.18 | buster-wikimedia | amd64 conda-analytics | 0.0.18 | bullseye-wikimedia | amd64 btullis@apt1001:~$ sudo -i reprepro includedeb buster-wikimedia `pwd`/conda-analytics-0.0.19_amd64.deb Exporting indices... btullis@apt1001:~$ sudo -i reprepro includedeb bullseye-wikimedia `pwd`/conda-analytics-0.0.19_amd64.deb Exporting indices... Deleting files no longer referenced... btullis@apt1001:~$ sudo -i reprepro ls conda-analytics conda-analytics | 0.0.19 | buster-wikimedia | amd64 conda-analytics | 0.0.19 | bullseye-wikimedia | amd64 btullis@apt1001:~$
I will start by installing it to the test cluster, then verifying that jupyter works and spark3 works.
Updated the apt repositories on the test cluster with:
btullis@cumin1001:~$ sudo cumin A:hadoop-test 'apt-get update'
Pushed out the new version of conda-analytics with:
btullis@cumin1001:~$ sudo debdeploy deploy -u 2023-09-13-conda-analytics.yaml -s hadoop-test
It's a good start, showing that I was able to activate the base environment and that the mamba solver appears in the system-wide config file as expected.
btullis@an-test-client1002:~$ source /usr/bin/conda-analytics-activate base
(base) btullis@an-test-client1002:~$ conda info
active environment : base
active env location : /opt/conda-analytics
shell level : 1
user config file : /home/btullis/.condarc
populated config files : /opt/conda-analytics/condarc
conda version : 23.5.2
conda-build version : not installed
python version : 3.10.8.final.0
virtual packages : __archspec=1=x86_64
__glibc=2.31=0
__linux=5.10.0=0
__unix=0=0
base environment : /opt/conda-analytics (read only)
conda av data dir : /opt/conda-analytics/etc/conda
conda av metadata url : None
channel URLs : https://conda.anaconda.org/conda-forge/linux-64
https://conda.anaconda.org/conda-forge/noarch
https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /home/btullis/.conda/pkgs
/opt/conda-analytics/pkgs
envs directories : /home/btullis/.conda/envs
/opt/conda-analytics/envs
platform : linux-64
user-agent : conda/23.5.2 requests/2.31.0 CPython/3.10.8 Linux/5.10.0-23-amd64 debian/11 glibc/2.31 solver/libmamba
UID:GID : 32741:500
netrc file : None
offline mode : False
(base) btullis@an-test-client1002:~$ tail -n 1 /opt/conda-analytics/condarc
solver: libmambaNext I will try cloning a new conda-analytics environment in a new jupyterhub session and installing something.
OK, less good results from jupyterhub.
First of all, I just tried connecting to the jupyterhub service that was already running.
That resulted in the following error:
btullis@an-test-client1002:~$ journalctl -u jupyter-btullis-singleuser-conda-analytics.service -- Journal begins at Wed 2023-09-13 14:02:06 UTC, ends at Wed 2023-09-13 16:29:04 UTC. -- Sep 13 16:26:33 an-test-client1002 systemd[1]: Started /bin/bash -c cd /home/btullis && exec /etc/jupyterhub-conda/jupyterhub-singleuser-conda-env.sh __NEW__ --port=58249 --SingleUserNotebookApp.default_url=/la> Sep 13 16:26:33 an-test-client1002 jupyterhub-conda-singleuser[3788386]: Creating new cloned conda env 2023-09-13T16.26.33_btullis... Sep 13 16:26:38 an-test-client1002 jupyterhub-conda-singleuser[3788390]: Source: /opt/conda-analytics Sep 13 16:26:38 an-test-client1002 jupyterhub-conda-singleuser[3788390]: Destination: /home/btullis/.conda/envs/2023-09-13T16.26.33_btullis Sep 13 16:26:38 an-test-client1002 jupyterhub-conda-singleuser[3788390]: The following packages cannot be cloned out of the root environment: Sep 13 16:26:38 an-test-client1002 jupyterhub-conda-singleuser[3788390]: - conda-forge/linux-64::conda-23.5.2-py310hff52083_0 Sep 13 16:26:38 an-test-client1002 jupyterhub-conda-singleuser[3788390]: Packages: 206 Sep 13 16:26:38 an-test-client1002 jupyterhub-conda-singleuser[3788390]: Files: 2709 Sep 13 16:26:38 an-test-client1002 jupyterhub-conda-singleuser[3788390]: Downloading and Extracting Packages Sep 13 16:26:38 an-test-client1002 jupyterhub-conda-singleuser[3788390]: Sep 13 16:26:38 an-test-client1002 jupyterhub-conda-singleuser[3788390]: Downloading and Extracting Packages Sep 13 16:26:38 an-test-client1002 jupyterhub-conda-singleuser[3788390]: Sep 13 16:26:40 an-test-client1002 jupyterhub-conda-singleuser[3788390]: Preparing transaction: ...working... done Sep 13 16:26:47 an-test-client1002 jupyterhub-conda-singleuser[3788390]: Verifying transaction: ...working... done Sep 13 16:27:18 an-test-client1002 jupyterhub-conda-singleuser[3788390]: Executing transaction: ...working... done Sep 13 16:27:18 an-test-client1002 jupyterhub-conda-singleuser[3788390]: # Sep 13 16:27:18 an-test-client1002 jupyterhub-conda-singleuser[3788390]: # To activate this environment, use Sep 13 16:27:18 an-test-client1002 jupyterhub-conda-singleuser[3788390]: # Sep 13 16:27:18 an-test-client1002 jupyterhub-conda-singleuser[3788390]: # $ conda activate 2023-09-13T16.26.33_btullis Sep 13 16:27:18 an-test-client1002 jupyterhub-conda-singleuser[3788390]: # Sep 13 16:27:18 an-test-client1002 jupyterhub-conda-singleuser[3788390]: # To deactivate an active environment, use Sep 13 16:27:18 an-test-client1002 jupyterhub-conda-singleuser[3788390]: # Sep 13 16:27:18 an-test-client1002 jupyterhub-conda-singleuser[3788390]: # $ conda deactivate Sep 13 16:27:19 an-test-client1002 jupyterhub-conda-singleuser[3788555]: CondaValueError: You have chosen a non-default solver backend (libmamba) but it was not recognized. Choose one of: classic Sep 13 16:27:19 an-test-client1002 systemd[1]: jupyter-btullis-singleuser-conda-analytics.service: Main process exited, code=exited, status=1/FAILURE Sep 13 16:27:19 an-test-client1002 systemd[1]: jupyter-btullis-singleuser-conda-analytics.service: Failed with result 'exit-code'. Sep 13 16:27:19 an-test-client1002 systemd[1]: jupyter-btullis-singleuser-conda-analytics.service: Consumed 32.984s CPU time.
This is understandable, since the jupyterhub service itself was still running out of the old conda-analytics-0.0.18 environment.
I checked that with:
btullis@an-test-client1002:~$ systemctl status jupyterhub-conda.service
● jupyterhub-conda.service - JupyterHub (conda)
Loaded: loaded (/lib/systemd/system/jupyterhub-conda.service; enabled; vendor preset: enabled)
Active: active (running) since Thu 2023-08-10 09:05:19 UTC; 1 months 3 days ago
Main PID: 491 (python)
Tasks: 8 (limit: 4661)
Memory: 102.6M
CPU: 47min 39.895s
CGroup: /system.slice/jupyterhub-conda.service
├─ 491 /opt/conda-analytics/bin/python /opt/conda-analytics/bin/jupyterhub --config=/etc/jupyterhub-conda/jupyterhub_config.py --no-ssl
└─3772 /opt/conda-analytics/bin/node /opt/conda-analytics/bin/configurable-http-proxy --ip 0.0.0.0 --port 8880 --api-ip 127.0.0.1 --api-port 8882 --error-target http://10.64.36.12:8881/hub/errorI then went to restart the service and then got an error restarting the service.
(base) btullis@an-test-client1002:~$ journalctl -u jupyterhub-conda.service -f -- Journal begins at Wed 2023-09-13 14:02:06 UTC. -- Sep 13 16:31:42 an-test-client1002 jupyterhub-conda[3789737]: sqlalchemy.exc.ArgumentError: Column expression, FROM clause, or other columns clause element expected, got [1]. Did you mean to say select(1)? Sep 13 16:31:42 an-test-client1002 jupyterhub-conda[3789737]: Sep 13 16:31:42 an-test-client1002 jupyterhub-conda[3789737]: [D 2023-09-13 16:31:42.954 JupyterHub application:1028] Exiting application: jupyterhub Sep 13 16:31:43 an-test-client1002 systemd[1]: jupyterhub-conda.service: Main process exited, code=exited, status=1/FAILURE Sep 13 16:31:43 an-test-client1002 systemd[1]: jupyterhub-conda.service: Failed with result 'exit-code'. Sep 13 16:31:43 an-test-client1002 systemd[1]: jupyterhub-conda.service: Consumed 1.193s CPU time. Sep 13 16:31:45 an-test-client1002 systemd[1]: jupyterhub-conda.service: Scheduled restart job, restart counter is at 12. Sep 13 16:31:45 an-test-client1002 systemd[1]: Stopped JupyterHub (conda). Sep 13 16:31:45 an-test-client1002 systemd[1]: jupyterhub-conda.service: Consumed 1.193s CPU time. Sep 13 16:31:45 an-test-client1002 systemd[1]: Started JupyterHub (conda). Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: [I 2023-09-13 16:31:46.205 JupyterHub app:2479] Running JupyterHub version 1.5.0 Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: [I 2023-09-13 16:31:46.206 JupyterHub app:2509] Using Authenticator: builtins.PosixGroupCheckingAuthenticator Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: [I 2023-09-13 16:31:46.207 JupyterHub app:2509] Using Spawner: spawners.CondaEnvProfilesSpawner Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: [I 2023-09-13 16:31:46.207 JupyterHub app:2509] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-1.5.0 Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: [I 2023-09-13 16:31:46.213 JupyterHub app:1554] Loading cookie_secret from /srv/jupyterhub-conda/data/jupyterhub_cookie_secret Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: [D 2023-09-13 16:31:46.214 JupyterHub app:1721] Connecting to db: sqlite:////srv/jupyterhub-conda/data/jupyterhub.sqlite.db Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: [E 2023-09-13 16:31:46.225 JupyterHub app:2989] Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: Traceback (most recent call last): Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/jupyterhub/app.py", line 2986, in launch_instance_async Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: await self.initialize(argv) Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/jupyterhub/app.py", line 2521, in initialize Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: self.init_db() Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/jupyterhub/app.py", line 1726, in init_db Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: self.session_factory = orm.new_session_factory( Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/jupyterhub/orm.py", line 880, in new_session_factory Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: check_db_revision(engine) Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/jupyterhub/orm.py", line 771, in check_db_revision Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: current_table_names = set(inspect(engine).get_table_names()) Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/inspection.py", line 145, in inspect Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: ret = reg(subject) Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/engine/reflection.py", line 303, in _engine_insp Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: return Inspector._construct(Inspector._init_engine, bind) Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/engine/reflection.py", line 236, in _construct Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: init(self, bind) Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/engine/reflection.py", line 247, in _init_engine Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: engine.connect().close() Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3264, in connect Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: return self._connection_cls(self) Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 174, in __init__ Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: self.dispatch.engine_connect(self) Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/event/attr.py", line 487, in __call__ Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: fn(*args, **kw) Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/event/legacy.py", line 99, in wrap_leg Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: return fn(*conv(*args)) Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/jupyterhub/orm.py", line 737, in ping_connection Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: connection.scalar(select([1])) Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/sql/_selectable_constructors.py", line 489, in select Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: return Select(*entities) Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/sql/selectable.py", line 5130, in __init__ Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: self._raw_columns = [ Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/sql/selectable.py", line 5131, in <listcomp> Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: coercions.expect( Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 413, in expect Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: resolved = impl._literal_coercion( Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 651, in _literal_coercion Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: self._raise_for_expected(element, argname) Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 1139, in _raise_for_expected Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: return super()._raise_for_expected( Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 710, in _raise_for_expected Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: super()._raise_for_expected( Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: File "/opt/conda-analytics/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 535, in _raise_for_expected Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: raise exc.ArgumentError(msg, code=code) from err Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: sqlalchemy.exc.ArgumentError: Column expression, FROM clause, or other columns clause element expected, got [1]. Did you mean to say select(1)? Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: Sep 13 16:31:46 an-test-client1002 jupyterhub-conda[3789750]: [D 2023-09-13 16:31:46.231 JupyterHub application:1028] Exiting application: jupyterhub
This is another SQLAlchemy error - just like the one we saw in T337471: Jupyterhub is broken in conda-analytics 0.0.13
It looks lik the sqlalchemy python package must have been updated again by mistake. I'll have a look at building another version.
btullis opened https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/32
Pin the version of sqlachemy used in conda-analytics
btullis merged https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/32
Pin the version of sqlachemy used in conda-analytics
I've now added conda-analytics version 0.0.20 to our apt repository.
btullis@apt1001:~$ wget https://gitlab.wikimedia.org/api/v4/projects/359/packages/generic/conda-analytics/0.0.20/conda-analytics-0.0.20_amd64.deb --2023-09-14 11:25:47-- https://gitlab.wikimedia.org/api/v4/projects/359/packages/generic/conda-analytics/0.0.20/conda-analytics-0.0.20_amd64.deb Resolving gitlab.wikimedia.org (gitlab.wikimedia.org)... 2620:0:861:2:208:80:154:145, 208.80.154.145 Connecting to gitlab.wikimedia.org (gitlab.wikimedia.org)|2620:0:861:2:208:80:154:145|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1031951288 (984M) [application/octet-stream] Saving to: ‘conda-analytics-0.0.20_amd64.deb’ conda-analytics-0.0.20_amd64.deb 100%[=====================================================================================================================>] 984.14M 110MB/s in 9.0s 2023-09-14 11:25:56 (110 MB/s) - ‘conda-analytics-0.0.20_amd64.deb’ saved [1031951288/1031951288] btullis@apt1001:~$ sudo -i reprepro ls conda-analytics conda-analytics | 0.0.19 | buster-wikimedia | amd64 conda-analytics | 0.0.19 | bullseye-wikimedia | amd64 btullis@apt1001:~$ sudo -i reprepro includedeb buster-wikimedia `pwd`/conda-analytics-0.0.20_amd64.deb Exporting indices... btullis@apt1001:~$ sudo -i reprepro includedeb bullseye-wikimedia `pwd`/conda-analytics-0.0.20_amd64.deb Exporting indices... Deleting files no longer referenced... btullis@apt1001:~$ sudo -i reprepro ls conda-analytics conda-analytics | 0.0.20 | buster-wikimedia | amd64 conda-analytics | 0.0.20 | bullseye-wikimedia | amd64 btullis@apt1001:~$
I'll try again on the test cluster.
Mentioned in SAL (#wikimedia-analytics) [2023-09-14T11:42:28Z] <btullis> deploying conda-analytics version 0.0.20 to the test cluster for T337258
It's still not right, unfortunately.
When I try to clone the environment I get the following message.
btullis@an-test-client1002:~$ conda-analytics-clone Creating new cloned conda env 2023-09-14T12.35.54_btullis... Source: /opt/conda-analytics Destination: /home/btullis/.conda/envs/2023-09-14T12.35.54_btullis The following packages cannot be cloned out of the root environment: - conda-forge/linux-64::conda-23.5.2-py310hff52083_0 Packages: 206 Files: 2709 Downloading and Extracting Packages Downloading and Extracting Packages Preparing transaction: done Verifying transaction: done Executing transaction: done # # To activate this environment, use # # $ conda activate 2023-09-14T12.35.54_btullis # # To deactivate an active environment, use # # $ conda deactivate CondaValueError: You have chosen a non-default solver backend (libmamba) but it was not recognized. Choose one of: classic
However, it does successfully clone the environment and I can activate it.
btullis@an-test-client1002:~$ conda activate 2023-09-14T12.35.54_btullis
(2023-09-14T12.35.54_btullis) btullis@an-test-client1002:~$ conda -V
conda 23.5.2
(2023-09-14T12.35.54_btullis) btullis@an-test-client1002:~$ conda info
active environment : 2023-09-14T12.35.54_btullis
active env location : /home/btullis/.conda/envs/2023-09-14T12.35.54_btullis
shell level : 1
user config file : /home/btullis/.condarc
populated config files : /opt/conda-analytics/condarc
/home/btullis/.conda/envs/2023-09-14T12.35.54_btullis/condarc
conda version : 23.5.2
conda-build version : not installed
python version : 3.10.8.final.0
virtual packages : __archspec=1=x86_64
__glibc=2.31=0
__linux=5.10.0=0
__unix=0=0
base environment : /opt/conda-analytics (read only)
conda av data dir : /opt/conda-analytics/etc/conda
conda av metadata url : None
channel URLs : https://conda.anaconda.org/conda-forge/linux-64
https://conda.anaconda.org/conda-forge/noarch
https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /home/btullis/.conda/pkgs
/home/btullis/.conda/envs/2023-09-14T12.35.54_btullis/pkgs
/opt/conda-analytics/pkgs
envs directories : /home/btullis/.conda/envs
/opt/conda-analytics/envs
platform : linux-64
user-agent : conda/23.5.2 requests/2.31.0 CPython/3.10.8 Linux/5.10.0-23-amd64 debian/11 glibc/2.31 solver/libmamba
UID:GID : 32741:500
netrc file : None
offline mode : False
(2023-09-14T12.35.54_btullis) btullis@an-test-client1002:~$Continuing to investigate.
btullis opened https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/33
Add conda-libmamba-solver to the conda-analytics environment
btullis merged https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/33
Add conda-libmamba-solver to the conda-analytics environment
Mentioned in SAL (#wikimedia-analytics) [2023-09-14T21:31:30Z] <btullis> deploying conda-analytics version 0.0.21 to hadoop-test for T337258
Hmm. Not the best result.
Firstly, it complains of a missing library when trying to list environments.
btullis@an-test-client1002:~$ conda-analytics-list Could not load conda plugin `conda-libmamba-solver`: libarchive.so.19: cannot open shared object file: No such file or directory WARNING conda.plugins.manager:load_entrypoints(85): Could not load conda plugin `conda-libmamba-solver`: libarchive.so.19: cannot open shared object file: No such file or directory # conda environments: # 2023-09-13T16.26.33_btullis /home/btullis/.conda/envs/2023-09-13T16.26.33_btullis 2023-09-14T12.25.12_btullis /home/btullis/.conda/envs/2023-09-14T12.25.12_btullis 2023-09-14T12.28.09_btullis /home/btullis/.conda/envs/2023-09-14T12.28.09_btullis 2023-09-14T12.35.54_btullis /home/btullis/.conda/envs/2023-09-14T12.35.54_btullis btullis-test /home/btullis/.conda/envs/btullis-test base /opt/conda-analytics
Then when trying to clone an environment, it still says that it's unavailable.
btullis@an-test-client1002:~$ conda-analytics-clone btullis-test2 Creating new cloned conda env btullis-test2... Could not load conda plugin `conda-libmamba-solver`: libarchive.so.19: cannot open shared object file: No such file or directory Source: /opt/conda-analytics Destination: /home/btullis/.conda/envs/btullis-test2 The following packages cannot be cloned out of the root environment: - conda-forge/linux-64::conda-23.5.2-py310hff52083_0 - conda-forge/noarch::conda-libmamba-solver-23.7.0-pyhd8ed1ab_0 Packages: 217 Files: 2709 Downloading and Extracting Packages Downloading and Extracting Packages Preparing transaction: done Verifying transaction: done Executing transaction: done # # To activate this environment, use # # $ conda activate btullis-test2 # # To deactivate an active environment, use # # $ conda deactivate Could not load conda plugin `conda-libmamba-solver`: libarchive.so.19: cannot open shared object file: No such file or directory CondaValueError: You have chosen a non-default solver backend (libmamba) but it was not recognized. Choose one of: classic
It's intriguing that libarchive.so.13 is available, but also it looks like the package for libarchive.so.19 is available. So why isn't it installed?
btullis@an-test-client1002:/opt/conda-analytics$ find . -name '*libarchive.so*' ./pkgs/libarchive-3.6.2-h6ac8c49_2/lib/libarchive.so.19 ./pkgs/libarchive-3.6.2-h6ac8c49_2/lib/libarchive.so ./pkgs/libarchive-3.6.2-hc8874e4_0/lib/libarchive.so.13 ./pkgs/libarchive-3.6.2-hc8874e4_0/lib/libarchive.so ./pkgs/libarchive-3.6.2-hc8874e4_0/lib/libarchive.so.13.6.2 ./lib/libarchive.so.13 ./lib/libarchive.so ./lib/libarchive.so.13.6.2
btullis opened https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/34
Downgrade the libmamba package so that it matches conda
btullis merged https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/34
Downgrade the libmamba package so that it matches conda
I tried a version of conda-analytics where the versions of conda and the conda-libmamba-solver were closer.
It's still not working. This is trying to spawn a new environment for jupyterhub.
Sep 15 14:10:17 an-test-client1002 systemd[1]: Started /bin/bash -c cd /home/btullis && exec /etc/jupyterhub-conda/jupyterhub-singleuser-conda-env.sh __NEW__ --port=46817 --SingleUserNotebookApp.default_url=/lab. Sep 15 14:10:17 an-test-client1002 jupyterhub-conda-singleuser[843023]: Creating new cloned conda env 2023-09-15T14.10.17_btullis... Sep 15 14:10:17 an-test-client1002 jupyterhub-conda-singleuser[843027]: Could not load conda plugin `conda-libmamba-solver`: Sep 15 14:10:17 an-test-client1002 jupyterhub-conda-singleuser[843027]: libarchive.so.19: cannot open shared object file: No such file or directory Sep 15 14:10:24 an-test-client1002 jupyterhub-conda-singleuser[843027]: Source: /opt/conda-analytics Sep 15 14:10:24 an-test-client1002 jupyterhub-conda-singleuser[843027]: Destination: /home/btullis/.conda/envs/2023-09-15T14.10.17_btullis Sep 15 14:10:24 an-test-client1002 jupyterhub-conda-singleuser[843027]: The following packages cannot be cloned out of the root environment: Sep 15 14:10:24 an-test-client1002 jupyterhub-conda-singleuser[843027]: - conda-forge/linux-64::conda-23.5.2-py310hff52083_0 Sep 15 14:10:24 an-test-client1002 jupyterhub-conda-singleuser[843027]: - conda-forge/noarch::conda-libmamba-solver-23.5.0-pyhd8ed1ab_0 Sep 15 14:10:24 an-test-client1002 jupyterhub-conda-singleuser[843027]: Packages: 217 Sep 15 14:10:24 an-test-client1002 jupyterhub-conda-singleuser[843027]: Files: 2782 Sep 15 14:10:24 an-test-client1002 jupyterhub-conda-singleuser[843027]: Downloading and Extracting Packages Sep 15 14:10:24 an-test-client1002 jupyterhub-conda-singleuser[843027]: Sep 15 14:10:24 an-test-client1002 jupyterhub-conda-singleuser[843027]: Downloading and Extracting Packages Sep 15 14:10:24 an-test-client1002 jupyterhub-conda-singleuser[843027]: Sep 15 14:10:25 an-test-client1002 jupyterhub-conda-singleuser[843027]: Preparing transaction: ...working... done Sep 15 14:10:34 an-test-client1002 jupyterhub-conda-singleuser[843027]: Verifying transaction: ...working... done Sep 15 14:11:08 an-test-client1002 jupyterhub-conda-singleuser[843027]: Executing transaction: ...working... done Sep 15 14:11:08 an-test-client1002 jupyterhub-conda-singleuser[843027]: # Sep 15 14:11:08 an-test-client1002 jupyterhub-conda-singleuser[843027]: # To activate this environment, use Sep 15 14:11:08 an-test-client1002 jupyterhub-conda-singleuser[843027]: # Sep 15 14:11:08 an-test-client1002 jupyterhub-conda-singleuser[843027]: # $ conda activate 2023-09-15T14.10.17_btullis Sep 15 14:11:08 an-test-client1002 jupyterhub-conda-singleuser[843027]: # Sep 15 14:11:08 an-test-client1002 jupyterhub-conda-singleuser[843027]: # To deactivate an active environment, use Sep 15 14:11:08 an-test-client1002 jupyterhub-conda-singleuser[843027]: # Sep 15 14:11:08 an-test-client1002 jupyterhub-conda-singleuser[843027]: # $ conda deactivate Sep 15 14:11:08 an-test-client1002 jupyterhub-conda-singleuser[843295]: Could not load conda plugin `conda-libmamba-solver`: Sep 15 14:11:08 an-test-client1002 jupyterhub-conda-singleuser[843295]: libarchive.so.19: cannot open shared object file: No such file or directory Sep 15 14:11:09 an-test-client1002 jupyterhub-conda-singleuser[843295]: CondaValueError: You have chosen a non-default solver backend (libmamba) but it was not recognized. Choose one of: classic Sep 15 14:11:09 an-test-client1002 systemd[1]: jupyter-btullis-singleuser-conda-analytics.service: Main process exited, code=exited, status=1/FAILURE Sep 15 14:11:09 an-test-client1002 systemd[1]: jupyter-btullis-singleuser-conda-analytics.service: Failed with result 'exit-code'. Sep 15 14:11:09 an-test-client1002 systemd[1]: jupyter-btullis-singleuser-conda-analytics.service: Consumed 36.167s CPU time.
This is just trying to list the conda environments.
btullis@an-test-client1002:~$ conda-analytics-list Could not load conda plugin `conda-libmamba-solver`: libarchive.so.19: cannot open shared object file: No such file or directory WARNING conda.plugins.manager:load_entrypoints(85): Could not load conda plugin `conda-libmamba-solver`: libarchive.so.19: cannot open shared object file: No such file or directory # conda environments: # 2023-09-13T16.26.33_btullis /home/btullis/.conda/envs/2023-09-13T16.26.33_btullis 2023-09-14T12.25.12_btullis /home/btullis/.conda/envs/2023-09-14T12.25.12_btullis 2023-09-14T12.28.09_btullis /home/btullis/.conda/envs/2023-09-14T12.28.09_btullis 2023-09-14T12.35.54_btullis /home/btullis/.conda/envs/2023-09-14T12.35.54_btullis 2023-09-15T14.10.17_btullis /home/btullis/.conda/envs/2023-09-15T14.10.17_btullis btullis-test /home/btullis/.conda/envs/btullis-test btullis-test2 /home/btullis/.conda/envs/btullis-test2 base /opt/conda-analytics
@BTullis I just released Wmfdata 2.0.1 with a few small fixes (the most important is pinning Urllib3, but I know you already did that manually).
Nothing at all urgent, but if it's easy to build a new conda-analytics including the new version, you might want to do that.
btullis opened https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/35
Try again to get the mamba solver working
I've tried two more builds, but I'm still finding the same issue. I think I'm closing in on what the problem is though.
There seem to be two different builds of libarchive in the pkgs directory. They are both libarchive-3.6.2 but they include different libraries.
(base) btullis@an-test-client1002:/opt/conda-analytics/pkgs$ tree libarchive-3.6.2-h6ac8c49_2/lib libarchive-3.6.2-hc8874e4_0/lib/
libarchive-3.6.2-h6ac8c49_2/lib
├── libarchive.a
├── libarchive.so -> libarchive.so.19
├── libarchive.so.19
└── pkgconfig
└── libarchive.pc
libarchive-3.6.2-hc8874e4_0/lib/
├── libarchive.a
├── libarchive.so -> libarchive.so.13.6.2
├── libarchive.so.13 -> libarchive.so.13.6.2
├── libarchive.so.13.6.2
└── pkgconfig
└── libarchive.pc
2 directories, 9 files
(base) btullis@an-test-client1002:/opt/conda-analytics/pkgs$The second one (libarchive-3.6.2-hc8874e4_0) is the one that is available con conda-forge. https://anaconda.org/conda-forge/libarchive/files
I haven't found the source of the first one yet, but I'm sure that I'm onto something.
Ahah! The different version comes from https://repo.anaconda.com/pkgs/main/linux-64/
So somewhere we are installing this from the main channel and then replacing it with the same version of the package from the conda-forge channel.
Interesting!
One observation:
We lock down the conda-environment.yml to only consider conda-forge on https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/blob/7d3115d59abc377f57e17b0e58be016de1ef462f/conda-environment.yml#L2.
But, for some reason, the conda-environment.lock.yml shows conda-forge and defaults as potential channels?: https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/blob/7d3115d59abc377f57e17b0e58be016de1ef462f/conda-environment.lock.yml#L3-4.
I've tried several more builds locally and I feel that I'm getting closer, but I'm still not quite there yet.
I tried some things based on the observations from @xcollazo above (thank you).
I really thought this last one was going to do it, but with all of these attempts I still ended up with remnants of both builds of libarchive in the tarball and the same error message when running conda-environment-list from the deployed environment.
For now I'm just going to make some more notes here, so apologies for the verbosity.
By stepping through manually the commands in the Dockerfile in a test container, I have verified that the libarchive.so.19 build from the main channel is installed to the /opt/miniconda build environment by the miniconda installer.
root@01237b0b7310:/# /tmp/minconda_installer.sh -b -p /opt/miniconda
PREFIX=/opt/miniconda
Unpacking payload ...
Installing base environment...
Downloading and Extracting Packages
Downloading and Extracting Packages
Preparing transaction: done
Executing transaction: done
installation finished.
root@01237b0b7310:/# ls -l /opt/miniconda/lib/libarchive*
-rw-rw-r-- 2 root root 1920398 Jun 2 23:14 /opt/miniconda/lib/libarchive.a
lrwxrwxrwx 1 root root 16 Sep 22 10:14 /opt/miniconda/lib/libarchive.so -> libarchive.so.19
-rwxrwxr-x 2 root root 917792 Jun 2 23:14 /opt/miniconda/lib/libarchive.so.19
root@01237b0b7310:/# ls -l /opt/miniconda/pkgs/libarchive*
-rw-r--r-- 1 root root 921262 Jul 13 18:43 /opt/miniconda/pkgs/libarchive-3.6.2-h6ac8c49_2.conda
/opt/miniconda/pkgs/libarchive-3.6.2-h6ac8c49_2:
total 20
drwxr-xr-x 2 root root 4096 Sep 22 10:14 bin
drwxr-xr-x 2 root root 4096 Sep 22 10:14 include
drwxr-xr-x 5 root root 4096 Sep 22 10:14 info
drwxr-xr-x 3 root root 4096 Sep 22 10:14 lib
drwxr-xr-x 3 root root 4096 Sep 22 10:14 share
root@01237b0b7310:/# ls -l /opt/miniconda/pkgs/libarchive*/lib
total 2780
-rw-rw-r-- 2 root root 1920398 Jun 2 23:14 libarchive.a
lrwxrwxrwx 1 root root 16 Sep 22 10:14 libarchive.so -> libarchive.so.19
-rwxrwxr-x 2 root root 917792 Jun 2 23:14 libarchive.so.19
drwxr-xr-x 2 root root 4096 Sep 22 10:14 pkgconfig
root@01237b0b7310:/#When I activate the build environment /opt/miniconda I get the following results from conda list and pip freeze respectively.
root@01237b0b7310:/srv/conda-analytics# source /opt/miniconda/bin/activate (base) root@01237b0b7310:/srv/conda-analytics# conda list # packages in environment at /opt/miniconda: # # Name Version Build Channel _libgcc_mutex 0.1 main _openmp_mutex 5.1 1_gnu boltons 23.0.0 py310h06a4308_0 brotlipy 0.7.0 py310h7f8727e_1002 bzip2 1.0.8 h7b6447c_0 c-ares 1.19.0 h5eee18b_0 ca-certificates 2023.05.30 h06a4308_0 certifi 2023.5.7 py310h06a4308_0 cffi 1.15.1 py310h5eee18b_3 charset-normalizer 2.0.4 pyhd3eb1b0_0 conda 23.5.2 py310h06a4308_0 conda-content-trust 0.1.3 py310h06a4308_0 conda-libmamba-solver 23.5.0 py310h06a4308_0 conda-package-handling 2.1.0 py310h06a4308_0 conda-package-streaming 0.8.0 py310h06a4308_0 cryptography 39.0.1 py310h9ce1e76_2 fmt 9.1.0 hdb19cb5_0 icu 58.2 he6710b0_3 idna 3.4 py310h06a4308_0 jsonpatch 1.32 pyhd3eb1b0_0 jsonpointer 2.1 pyhd3eb1b0_0 krb5 1.20.1 h143b758_1 ld_impl_linux-64 2.38 h1181459_1 libarchive 3.6.2 h6ac8c49_2 libcurl 8.1.1 h251f7ec_1 libedit 3.1.20221030 h5eee18b_0 libev 4.33 h7f8727e_1 libffi 3.4.4 h6a678d5_0 libgcc-ng 11.2.0 h1234567_1 libgomp 11.2.0 h1234567_1 libmamba 1.4.1 h2dafd23_1 libmambapy 1.4.1 py310h2dafd23_1 libnghttp2 1.52.0 h2d74bed_1 libsolv 0.7.22 he621ea3_0 libssh2 1.10.0 hdbd6064_2 libstdcxx-ng 11.2.0 h1234567_1 libuuid 1.41.5 h5eee18b_0 libxml2 2.10.3 hcbfbd50_0 lz4-c 1.9.4 h6a678d5_0 ncurses 6.4 h6a678d5_0 openssl 3.0.9 h7f8727e_0 packaging 23.0 py310h06a4308_0 pcre2 10.37 he7ceb23_1 pip 23.1.2 py310h06a4308_0 pluggy 1.0.0 py310h06a4308_1 pybind11-abi 4 hd3eb1b0_1 pycosat 0.6.4 py310h5eee18b_0 pycparser 2.21 pyhd3eb1b0_0 pyopenssl 23.0.0 py310h06a4308_0 pysocks 1.7.1 py310h06a4308_0 python 3.10.12 h955ad1f_0 readline 8.2 h5eee18b_0 reproc 14.2.4 h295c915_1 reproc-cpp 14.2.4 h295c915_1 requests 2.29.0 py310h06a4308_0 ruamel.yaml 0.17.21 py310h5eee18b_0 ruamel.yaml.clib 0.2.6 py310h5eee18b_1 setuptools 67.8.0 py310h06a4308_0 six 1.16.0 pyhd3eb1b0_1 sqlite 3.41.2 h5eee18b_0 tk 8.6.12 h1ccaba5_0 toolz 0.12.0 py310h06a4308_0 tqdm 4.65.0 py310h2f386ee_0 tzdata 2023c h04d1e81_0 urllib3 1.26.16 py310h06a4308_0 wheel 0.38.4 py310h06a4308_0 xz 5.4.2 h5eee18b_0 yaml-cpp 0.7.0 h295c915_1 zlib 1.2.13 h5eee18b_0 zstandard 0.19.0 py310h5eee18b_0 zstd 1.5.5 hc292b87_0 (base) root@01237b0b7310:/srv/conda-analytics# (base) root@01237b0b7310:/srv/conda-analytics# pip freeze boltons @ file:///croot/boltons_1677628692245/work brotlipy==0.7.0 certifi @ file:///croot/certifi_1683875369620/work/certifi cffi @ file:///croot/cffi_1670423208954/work charset-normalizer @ file:///tmp/build/80754af9/charset-normalizer_1630003229654/work conda @ file:///croot/conda_1689269889729/work conda-content-trust @ file:///tmp/abs_5952f1c8-355c-4855-ad2e-538535021ba5h26t22e5/croots/recipe/conda-content-trust_1658126371814/work conda-libmamba-solver @ file:///croot/conda-libmamba-solver_1685032319139/work/src conda-package-handling @ file:///croot/conda-package-handling_1685024767917/work conda_package_streaming @ file:///croot/conda-package-streaming_1685019673878/work cryptography @ file:///croot/cryptography_1686613057838/work idna @ file:///croot/idna_1666125576474/work jsonpatch @ file:///tmp/build/80754af9/jsonpatch_1615747632069/work jsonpointer==2.1 libmambapy @ file:///croot/mamba-split_1685993156657/work/libmambapy packaging @ file:///croot/packaging_1678965309396/work pluggy @ file:///tmp/build/80754af9/pluggy_1648024709248/work pycosat @ file:///croot/pycosat_1666805502580/work pycparser @ file:///tmp/build/80754af9/pycparser_1636541352034/work pyOpenSSL @ file:///croot/pyopenssl_1677607685877/work PySocks @ file:///home/builder/ci_310/pysocks_1640793678128/work requests @ file:///croot/requests_1682607517574/work ruamel.yaml @ file:///croot/ruamel.yaml_1666304550667/work ruamel.yaml.clib @ file:///croot/ruamel.yaml.clib_1666302247304/work six @ file:///tmp/build/80754af9/six_1644875935023/work toolz @ file:///croot/toolz_1667464077321/work tqdm @ file:///croot/tqdm_1679561862951/work urllib3 @ file:///croot/urllib3_1686163155763/work zstandard @ file:///croot/zstandard_1677013143055/work (base) root@01237b0b7310:/srv/conda-analytics#
Observations:
Here is the conda info output for good measure.
(base) root@01237b0b7310:/srv/conda-analytics# conda info
active environment : base
active env location : /opt/miniconda
shell level : 1
user config file : /root/.condarc
populated config files :
conda version : 23.5.2
conda-build version : not installed
python version : 3.10.12.final.0
virtual packages : __archspec=1=x86_64
__glibc=2.28=0
__linux=6.2.0=0
__unix=0=0
base environment : /opt/miniconda (writable)
conda av data dir : /opt/miniconda/etc/conda
conda av metadata url : None
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /opt/miniconda/pkgs
/root/.conda/pkgs
envs directories : /opt/miniconda/envs
/root/.conda/envs
platform : linux-64
user-agent : conda/23.5.2 requests/2.29.0 CPython/3.10.12 Linux/6.2.0-33-generic debian/10 glibc/2.28
UID:GID : 0:0
netrc file : None
offline mode : False
(base) root@01237b0b7310:/srv/conda-analytics#This all correlates with the version of the miniconda installer which is currently specified in the Dockerfile, which is still the latest avaiable on the conda website and https://repo.anaconda.com/miniconda/
I don't know why we can't download an installer for conda 23.7.4 yet.
I was going to try a build with the order of the channels reversed in the conda-environment.lock.yaml as shown.
(base) root@01237b0b7310:/srv/conda-analytics# head -n 4 conda-environment.lock.yml name: base channels: - defaults - conda-forge
But then I read this: https://github.com/conda/conda/issues/8675 and this: https://stackoverflow.com/questions/58555389/set-the-channel-priority-in-conda-environment-yaml
...and they lead me to believe that setting channel priority via environment.yaml (and by extension environment.local.yaml) files is not possible.
One workaround suggested is to use a channel prefix conda-forge for each package that needs to come from a separate channel.
I suppose I have a question at this stage, which is why do we prefer conda-forge over the defaults channel (as is stated here)?
I might try building an environment from only the defaults channel and see if it works.
Remove conda-forge from the conda-environment.lock.yml manually.
(base) root@01237b0b7310:/srv/conda-analytics# head -n 4 conda-environment.lock.yml name: base channels: - defaults dependencies:
Attempt to build the environment.
(base) root@01237b0b7310:/srv/conda-analytics# conda env create --file conda-environment.lock.yml --name conda_dist_env Collecting package metadata (repodata.json): done Solving environment: failed ResolvePackageNotFound: - tornado=6.3.3 - ruamel.yaml=0.17.32 - anyio=3.7.1 - asttokens=2.4.0 - charset-normalizer=3.2.0 - argon2-cffi=23.1.0 - jupyterhub-systemdspawner=0.15.0 - jupyterlab_pygments=0.2.2 - zipp=3.17.0 - jupyterlab=3.4.8 - pluggy=1.3.0 - aws-c-io=0.10.5 - orc=1.7.5 - libgcc-ng=13.2.0 - liblapack=3.9.0 - prompt-toolkit=3.0.39 - greenlet=2.0.2 - markupsafe=2.1.3 - libzlib=1.2.13 - blinker=1.6.2 - re2=2022.06.01 - libopenblas=0.3.24 - jupyter_core=5.3.1 - websocket-client=1.6.3 - libiconv=1.17 - jedi=0.19.0 - nbconvert-pandoc=7.8.0 - libgfortran-ng=13.2.0 - wheel=0.41.2 - jupyter_server=1.24.0 - rpds-py=0.10.3 - zstandard=0.21.0 - jupyterhub-base=1.5.0 - s2n=1.0.10 - setuptools=68.2.2 - ca-certificates=2023.7.22 - nbclient=0.8.0 - executing=1.2.0 - notebook=6.5.6 - sniffio=1.3.0 - soupsieve=2.5 - backports.functools_lru_cache=1.6.5 - attrs=23.1.0 - jupyterhub=1.5.0 - json5=0.9.14 - jupyterlab_server=2.25.0 - notebook-shim=0.2.3 - python-fastjsonschema=2.18.0 - send2trash=1.8.2 - libsqlite=3.43.0 - mako=1.2.4 - pyjwt=2.8.0 - libgoogle-cloud=1.40.2 - prompt_toolkit=3.0.39 - nodejs=12.4.0 - libthrift=0.16.0 - libgomp=13.2.0 - python_abi=3.10 - keyutils=1.6.1 - comm=0.1.4 - aws-c-event-stream=0.2.7 - snappy=1.1.10 - jsonschema=4.19.1 - glog=0.6.0 - prometheus_client=0.17.1 - ld_impl_linux-64=2.40 - traitlets=5.10.0 - pyzmq=24.0.1 - libgfortran5=13.2.0 - pyasn1=0.5.0 - nbconvert-core=7.8.0 - typing_extensions=4.8.0 - aws-sdk-cpp=1.8.186 - configurable-http-proxy=4.2.3 - cryptography=39.0.0 - babel=2.12.1 - jupyterhub-singleuser=1.5.0 - icu=72.1 - importlib_resources=6.0.1 - ipykernel=6.25.2 - mistune=3.0.1 - debugpy=1.8.0 - pamela=1.1.0 - nbconvert=7.8.0 - bleach=6.0.0 - libstdcxx-ng=13.2.0 - sqlalchemy=1.4.49 - pandoc=3.1.3 - libblas=3.9.0 - libevent=2.1.10 - libcblas=3.9.0 - tqdm=4.66.1 - aws-c-cal=0.5.11 - pygments=2.16.1 - wcwidth=0.2.6 - libuuid=2.38.1 - stack_data=0.6.2 - psutil=5.9.5 - libutf8proc=2.8.0 - referencing=0.30.2 - typing-extensions=4.8.0 - exceptiongroup=1.1.3 - importlib-metadata=6.8.0 - nbclassic=1.0.0 - jsonschema-specifications=2023.7.1 - jsonpointer=2.4 - grpc-cpp=1.45.2 - alembic=1.12.0
OK, this is a pretty useful list to work with. I can see that a few of these simply don't exist in the defaults channel, whereas others are older versions.
e.g. jupyterhub-systemdspawner isn't present, jupyterhub-singleuser is only at version 1.3 in the defaults channel.
I'll reset the channels in the environment file and then build an environment in the usual way.
Created the environment.
(base) root@01237b0b7310:/srv/conda-analytics# head -n 4 conda-environment.lock.yml name: base channels: - conda-forge - defaults (base) root@01237b0b7310:/srv/conda-analytics# conda env create --file conda-environment.lock.yml --name conda_dist_env <snip snip> done # # To activate this environment, use # # $ conda activate conda_dist_env # # To deactivate an active environment, use # # $ conda deactivate (base) root@01237b0b7310:/srv/conda-analytics#
Activated the environment and ran conda list
(base) root@01237b0b7310:/srv/conda-analytics# conda activate conda_dist_env (conda_dist_env) root@01237b0b7310:/srv/conda-analytics#conda list # packages in environment at /opt/miniconda/envs/conda_dist_env: # # Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge abseil-cpp 20210324.2 h9c3ff4c_0 conda-forge alembic 1.12.0 pyhd8ed1ab_0 conda-forge anyio 3.7.1 pyhd8ed1ab_0 conda-forge argon2-cffi 23.1.0 pyhd8ed1ab_0 conda-forge argon2-cffi-bindings 21.2.0 py310h5764c6d_3 conda-forge arrow-cpp 8.0.0 py310hbe01597_5_cpu conda-forge asttokens 2.4.0 pyhd8ed1ab_0 conda-forge async_generator 1.10 py_0 conda-forge attrs 23.1.0 pyh71513ae_1 conda-forge aws-c-cal 0.5.11 h95a6274_0 conda-forge aws-c-common 0.6.2 h7f98852_0 conda-forge aws-c-event-stream 0.2.7 h3541f99_13 conda-forge aws-c-io 0.10.5 hfb6a706_0 conda-forge aws-checksums 0.1.11 ha31a3da_7 conda-forge aws-sdk-cpp 1.8.186 hecaee15_4 conda-forge babel 2.12.1 pyhd8ed1ab_1 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 pyhd8ed1ab_3 conda-forge backports.functools_lru_cache 1.6.5 pyhd8ed1ab_0 conda-forge beautifulsoup4 4.12.2 pyha770c72_0 conda-forge bleach 6.0.0 pyhd8ed1ab_0 conda-forge blinker 1.6.2 pyhd8ed1ab_0 conda-forge boltons 23.0.0 pyhd8ed1ab_0 conda-forge brotlipy 0.7.0 py310h5764c6d_1005 conda-forge bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.19.1 hd590300_0 conda-forge ca-certificates 2023.7.22 hbcca054_0 conda-forge certifi 2023.7.22 pyhd8ed1ab_0 conda-forge certipy 0.1.3 py_0 conda-forge cffi 1.15.1 py310h2fee648_5 conda-forge charset-normalizer 3.2.0 pyhd8ed1ab_0 conda-forge click 8.1.7 pypi_0 pypi colorama 0.4.6 pyhd8ed1ab_0 conda-forge comm 0.1.4 pyhd8ed1ab_0 conda-forge conda 23.7.4 py310hff52083_0 conda-forge conda-libmamba-solver 23.7.0 pyhd8ed1ab_0 conda-forge conda-pack 0+untagged.271.ge3a8735 pypi_0 pypi conda-package-handling 2.2.0 pyh38be061_0 conda-forge conda-package-streaming 0.9.0 pyhd8ed1ab_0 conda-forge configurable-http-proxy 4.2.3 node12_hac6e442_0 conda-forge cryptography 39.0.0 py310h65dfdc0_0 conda-forge debugpy 1.8.0 py310hc6cd4ac_0 conda-forge decorator 5.1.1 pyhd8ed1ab_0 conda-forge defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge entrypoints 0.4 pyhd8ed1ab_0 conda-forge exceptiongroup 1.1.3 pyhd8ed1ab_0 conda-forge executing 1.2.0 pyhd8ed1ab_0 conda-forge findspark 2.0.1 pypi_0 pypi fmt 9.1.0 h924138e_0 conda-forge future 0.18.3 pypi_0 pypi gflags 2.2.2 he1b5a44_1004 conda-forge glog 0.6.0 h6f12383_0 conda-forge gmp 6.2.1 h58526e2_0 conda-forge greenlet 2.0.2 py310hc6cd4ac_1 conda-forge grpc-cpp 1.45.2 h9d3bbbb_5 conda-forge gssapi 1.8.3 pypi_0 pypi icu 72.1 hcb278e6_0 conda-forge idna 3.4 pyhd8ed1ab_0 conda-forge importlib-metadata 6.8.0 pyha770c72_0 conda-forge importlib_resources 6.0.1 pyhd8ed1ab_0 conda-forge ipykernel 6.25.2 pyh2140261_0 conda-forge ipython 8.15.0 pyh0d859eb_0 conda-forge ipython_genutils 0.2.0 py_1 conda-forge jedi 0.19.0 pyhd8ed1ab_0 conda-forge jinja2 3.1.2 pyhd8ed1ab_1 conda-forge json5 0.9.14 pyhd8ed1ab_0 conda-forge jsonpatch 1.32 pyhd8ed1ab_0 conda-forge jsonpointer 2.4 py310hff52083_2 conda-forge jsonschema 4.19.1 pyhd8ed1ab_0 conda-forge jsonschema-specifications 2023.7.1 pyhd8ed1ab_0 conda-forge jupyter_client 7.4.9 pyhd8ed1ab_0 conda-forge jupyter_core 5.3.1 py310hff52083_0 conda-forge jupyter_server 1.24.0 pyhd8ed1ab_0 conda-forge jupyter_telemetry 0.1.0 pyhd8ed1ab_1 conda-forge jupyterhub 1.5.0 py310hff52083_1 conda-forge jupyterhub-base 1.5.0 py310hff52083_1 conda-forge jupyterhub-ldapauthenticator 1.3.2 py_0 conda-forge jupyterhub-singleuser 1.5.0 py310hff52083_1 conda-forge jupyterhub-systemdspawner 0.15.0 py310hff52083_3 conda-forge jupyterlab 3.4.8 pyhd8ed1ab_0 conda-forge jupyterlab_pygments 0.2.2 pyhd8ed1ab_0 conda-forge jupyterlab_server 2.25.0 pyhd8ed1ab_0 conda-forge keyutils 1.6.1 h166bdaf_0 conda-forge krb5 0.5.1 pypi_0 pypi ld_impl_linux-64 2.40 h41732ed_0 conda-forge ldap3 2.9.1 pyhd8ed1ab_0 conda-forge libarchive 3.6.2 hc8874e4_0 conda-forge libblas 3.9.0 18_linux64_openblas conda-forge libbrotlicommon 1.0.9 h166bdaf_9 conda-forge libbrotlidec 1.0.9 h166bdaf_9 conda-forge libbrotlienc 1.0.9 h166bdaf_9 conda-forge libcblas 3.9.0 18_linux64_openblas conda-forge libcrc32c 1.1.2 h9c3ff4c_0 conda-forge libcurl 8.2.1 h91b91d3_0 libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libevent 2.1.10 h9b69904_4 conda-forge libffi 3.4.2 h7f98852_5 conda-forge libgcc-ng 13.2.0 h807b86a_2 conda-forge libgfortran-ng 13.2.0 h69a702a_2 conda-forge libgfortran5 13.2.0 ha4646dd_2 conda-forge libgomp 13.2.0 h807b86a_2 conda-forge libgoogle-cloud 1.40.2 habd0e3a_0 conda-forge libiconv 1.17 h166bdaf_0 conda-forge liblapack 3.9.0 18_linux64_openblas conda-forge libmamba 1.5.1 hba0046a_0 libmambapy 1.5.1 py310ha06983f_0 libnghttp2 1.52.0 ha637b67_1 libnsl 2.0.0 h7f98852_0 conda-forge libopenblas 0.3.24 pthreads_h413a1c8_0 conda-forge libprotobuf 3.20.3 h3eb15da_0 conda-forge libsodium 1.0.18 h36c2ea0_1 conda-forge libsolv 0.7.24 hfc55251_4 conda-forge libsqlite 3.43.0 h2797004_0 conda-forge libssh2 1.10.0 haa6b8db_3 conda-forge libstdcxx-ng 13.2.0 h7e041cc_2 conda-forge libthrift 0.16.0 h491838f_2 conda-forge libutf8proc 2.8.0 h166bdaf_0 conda-forge libuuid 2.38.1 h0b41bf4_0 conda-forge libxml2 2.10.4 hfdac1af_0 conda-forge libzlib 1.2.13 hd590300_5 conda-forge lz4-c 1.9.4 hcb278e6_0 conda-forge lzo 2.10 h516909a_1000 conda-forge mako 1.2.4 pyhd8ed1ab_0 conda-forge mariadb 1.0.11 pypi_0 pypi markupsafe 2.1.3 py310h2372a71_1 conda-forge matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge mistune 3.0.1 pyhd8ed1ab_0 conda-forge nbclassic 1.0.0 pyhb4ecaf3_1 conda-forge nbclient 0.8.0 pyhd8ed1ab_0 conda-forge nbconvert 7.8.0 pyhd8ed1ab_0 conda-forge nbconvert-core 7.8.0 pyhd8ed1ab_0 conda-forge nbconvert-pandoc 7.8.0 pyhd8ed1ab_0 conda-forge nbformat 5.9.2 pyhd8ed1ab_0 conda-forge ncurses 6.4 hcb278e6_0 conda-forge nest-asyncio 1.5.6 pyhd8ed1ab_0 conda-forge nodejs 12.4.0 he1b5a44_0 conda-forge notebook 6.5.6 pyha770c72_0 conda-forge notebook-shim 0.2.3 pyhd8ed1ab_0 conda-forge numpy 1.23.1 py310h53a5b5f_0 conda-forge oauthlib 3.2.2 pyhd8ed1ab_0 conda-forge openssl 1.1.1w hd590300_0 conda-forge orc 1.7.5 h6c59b99_0 conda-forge packaging 23.1 pyhd8ed1ab_0 conda-forge pamela 1.1.0 pyh1a96a4e_0 conda-forge pandas 1.4.3 py310h769672d_0 conda-forge pandoc 3.1.3 h32600fe_0 conda-forge pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge parquet-cpp 1.5.1 2 conda-forge parso 0.8.3 pyhd8ed1ab_0 conda-forge pexpect 4.8.0 pyh1a96a4e_2 conda-forge pickleshare 0.7.5 py_1003 conda-forge pip 23.2.1 pyhd8ed1ab_0 conda-forge pkgutil-resolve-name 1.3.10 pyhd8ed1ab_1 conda-forge platformdirs 3.10.0 pyhd8ed1ab_0 conda-forge pluggy 1.3.0 pyhd8ed1ab_0 conda-forge presto-python-client 0.8.4 pypi_0 pypi prometheus_client 0.17.1 pyhd8ed1ab_0 conda-forge prompt-toolkit 3.0.39 pyha770c72_0 conda-forge prompt_toolkit 3.0.39 hd8ed1ab_0 conda-forge psutil 5.9.5 py310h2372a71_1 conda-forge ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge pure-sasl 0.6.2 pypi_0 pypi pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge py4j 0.10.9 pyh9f0ad1d_0 conda-forge pyarrow 8.0.0 py310h225c066_5_cpu conda-forge pyasn1 0.5.0 pyhd8ed1ab_0 conda-forge pybind11-abi 4 hd8ed1ab_3 conda-forge pycosat 0.6.4 py310h5764c6d_1 conda-forge pycparser 2.21 pyhd8ed1ab_0 conda-forge pycurl 7.45.1 py310h2aed498_3 conda-forge pygments 2.16.1 pyhd8ed1ab_0 conda-forge pyhive 0.7.0 pypi_0 pypi pyjwt 2.8.0 pyhd8ed1ab_0 conda-forge pyopenssl 23.2.0 pyhd8ed1ab_1 conda-forge pysocks 1.7.1 pyha2e5f31_6 conda-forge pyspark 3.1.2 pyh6c4a22f_0 conda-forge pyspnego 0.9.2 pypi_0 pypi python 3.10.8 h257c98d_0_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-fastjsonschema 2.18.0 pyhd8ed1ab_0 conda-forge python-json-logger 2.0.7 pyhd8ed1ab_0 conda-forge python_abi 3.10 4_cp310 conda-forge pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge pyzmq 24.0.1 py310h330234f_1 conda-forge re2 2022.06.01 h27087fc_1 conda-forge readline 8.2 h8228510_1 conda-forge referencing 0.30.2 pyhd8ed1ab_0 conda-forge reproc 14.2.4 h0b41bf4_0 conda-forge reproc-cpp 14.2.4 hcb278e6_0 conda-forge requests 2.31.0 pyhd8ed1ab_0 conda-forge requests-kerberos 0.14.0 pypi_0 pypi rpds-py 0.10.3 py310hcb5633a_0 conda-forge ruamel.yaml 0.17.32 py310h2372a71_0 conda-forge ruamel.yaml.clib 0.2.7 py310h1fa729e_1 conda-forge s2n 1.0.10 h9b69904_0 conda-forge sasl 0.3.1 pypi_0 pypi send2trash 1.8.2 pyh41d4057_0 conda-forge setuptools 68.2.2 pyhd8ed1ab_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge snappy 1.1.10 h9fff704_0 conda-forge sniffio 1.3.0 pyhd8ed1ab_0 conda-forge soupsieve 2.5 pyhd8ed1ab_1 conda-forge sqlalchemy 1.4.49 py310h2372a71_0 conda-forge stack_data 0.6.2 pyhd8ed1ab_0 conda-forge terminado 0.17.1 pyh41d4057_0 conda-forge thrift 0.16.0 pypi_0 pypi thrift-sasl 0.4.3 pypi_0 pypi tinycss2 1.2.1 pyhd8ed1ab_0 conda-forge tk 8.6.12 h27826a3_0 conda-forge tomli 2.0.1 pyhd8ed1ab_0 conda-forge toolz 0.12.0 pyhd8ed1ab_0 conda-forge tornado 6.3.3 py310h2372a71_1 conda-forge tqdm 4.66.1 pyhd8ed1ab_0 conda-forge traitlets 5.10.0 pyhd8ed1ab_0 conda-forge typing-extensions 4.8.0 hd8ed1ab_0 conda-forge typing_extensions 4.8.0 pyha770c72_0 conda-forge tzdata 2023c h71feb2d_0 conda-forge urllib3 1.26.15 pyhd8ed1ab_0 conda-forge wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge webencodings 0.5.1 pyhd8ed1ab_2 conda-forge websocket-client 1.6.3 pyhd8ed1ab_0 conda-forge wheel 0.41.2 pyhd8ed1ab_0 conda-forge wmfdata 2.0.1 pypi_0 pypi wrapspawner 1.0.1 pypi_0 pypi xz 5.2.6 h166bdaf_0 conda-forge yaml-cpp 0.7.0 h27087fc_2 conda-forge zeromq 4.3.4 h9c3ff4c_1 conda-forge zipp 3.17.0 pyhd8ed1ab_0 conda-forge zlib 1.2.13 hd590300_5 conda-forge zstandard 0.21.0 py310h1275a96_0 conda-forge zstd 1.5.5 hfc55251_0 conda-forge (conda_dist_env) root@01237b0b7310:/srv/conda-analytics#
Observations:
libcurl 8.2.1 h91b91d3_0 libmamba 1.5.1 hba0046a_0 libmambapy 1.5.1 py310ha06983f_0 libnghttp2 1.52.0 ha637b67_1
These 4 are all different builds from those in the /opt/miniconda environment though.
The output from conda info in this newly activated environment is also interesting.
(conda_dist_env) root@01237b0b7310:/srv/conda-analytics# conda info
active environment : conda_dist_env
active env location : /opt/miniconda/envs/conda_dist_env
shell level : 2
user config file : /root/.condarc
populated config files :
conda version : 23.5.2
conda-build version : not installed
python version : 3.10.12.final.0
virtual packages : __archspec=1=x86_64
__glibc=2.28=0
__linux=6.2.0=0
__unix=0=0
base environment : /opt/miniconda (writable)
conda av data dir : /opt/miniconda/etc/conda
conda av metadata url : None
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /opt/miniconda/pkgs
/root/.conda/pkgs
envs directories : /opt/miniconda/envs
/root/.conda/envs
platform : linux-64
user-agent : conda/23.5.2 requests/2.29.0 CPython/3.10.12 Linux/6.2.0-33-generic debian/10 glibc/2.28
UID:GID : 0:0
netrc file : None
offline mode : FalseIt says shell level : 2 and conda version : 23.5.2 whereas we can see that we have just installed conda version 23.7.4 into this new environment.
I will see what happens if I dectivate this set of two shell levels and try to activate this new environment directly.
Woah there! Earliest recorded occurrence of the libarchive error, running conda info at shell level 0. I was not expecting that.
(conda_dist_env) root@01237b0b7310:/srv/conda-analytics# source /opt/miniconda/envs/conda_dist_env/bin/deactivate
DeprecationWarning: 'source deactivate' is deprecated. Use 'conda deactivate'.
(/opt/miniconda) root@01237b0b7310:/srv/conda-analytics# conda deactivate
root@01237b0b7310:/srv/conda-analytics# conda info
Error while loading conda entry point: conda-libmamba-solver (libarchive.so.19: cannot open shared object file: No such file or directory)
active environment : None
shell level : 0
user config file : /root/.condarc
populated config files :
conda version : 23.7.4
conda-build version : not installed
python version : 3.10.8.final.0
virtual packages : __archspec=1=x86_64
__glibc=2.28=0
__linux=6.2.0=0
__unix=0=0
base environment : /opt/miniconda/envs/conda_dist_env (writable)
conda av data dir : /opt/miniconda/envs/conda_dist_env/etc/conda
conda av metadata url : None
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /opt/miniconda/envs/conda_dist_env/pkgs
/root/.conda/pkgs
envs directories : /opt/miniconda/envs/conda_dist_env/envs
/root/.conda/envs
platform : linux-64
user-agent : conda/23.7.4 requests/2.31.0 CPython/3.10.8 Linux/6.2.0-33-generic debian/10 glibc/2.28
UID:GID : 0:0
netrc file : None
offline mode : FalseAt last, I have a working version without the libarchive error.
btullis@an-test-client1002:~$ sudo dpkg -i conda-analytics-0.0.23.dev_amd64.deb (Reading database ... 294287 files and directories currently installed.) Preparing to unpack conda-analytics-0.0.23.dev_amd64.deb ... Unpacking conda-analytics (0.0.23) over (0.0.23) ... Setting up conda-analytics (0.0.23) ... Post install script. Running /opt/conda-analytics/bin/python /opt/conda-analytics/bin/conda-unpack... btullis@an-test-client1002:~$ conda-analytics-list # conda environments: # 2023-09-13T16.26.33_btullis /home/btullis/.conda/envs/2023-09-13T16.26.33_btullis 2023-09-14T12.25.12_btullis /home/btullis/.conda/envs/2023-09-14T12.25.12_btullis 2023-09-14T12.28.09_btullis /home/btullis/.conda/envs/2023-09-14T12.28.09_btullis 2023-09-14T12.35.54_btullis /home/btullis/.conda/envs/2023-09-14T12.35.54_btullis 2023-09-15T14.10.17_btullis /home/btullis/.conda/envs/2023-09-15T14.10.17_btullis 2023-09-21T16.18.06_btullis /home/btullis/.conda/envs/2023-09-21T16.18.06_btullis btullis-test /home/btullis/.conda/envs/btullis-test btullis-test2 /home/btullis/.conda/envs/btullis-test2 base /opt/conda-analytics btullis@an-test-client1002:~$
Still a little work to do on getting the conda-analytics-clone command to work.
btullis@an-test-client1002:~$ conda-analytics-clone
Creating new cloned conda env 2023-09-22T14.13.19_btullis...
Source: /opt/conda-analytics
Destination: /home/btullis/.conda/envs/2023-09-22T14.13.19_btullis
The following packages cannot be cloned out of the root environment:
- conda-forge/linux-64::conda-23.7.4-py310hff52083_0
- conda-forge/noarch::conda-libmamba-solver-23.7.0-pyhd8ed1ab_0
Packages: 213
Files: 963
Downloading and Extracting Packages
Downloading and Extracting Packages
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate 2023-09-22T14.13.19_btullis
#
# To deactivate an active environment, use
#
# $ conda deactivate
Channels:
- local
- conda-forge
- defaults
Platform: linux-64
Collecting package metadata (repodata.json): failed
# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
Traceback (most recent call last):
File "/opt/conda-analytics/lib/python3.10/site-packages/conda/exception_handler.py", line 17, in __call__
return func(*args, **kwargs)
File "/opt/conda-analytics/lib/python3.10/site-packages/conda/cli/main.py", line 64, in main_subshell
exit_code = do_call(args, parser)
File "/opt/conda-analytics/lib/python3.10/site-packages/conda/cli/conda_argparse.py", line 167, in do_call
result = getattr(module, func_name)(args, parser)
File "/opt/conda-analytics/lib/python3.10/site-packages/conda/notices/core.py", line 124, in wrapper
return func(*args, **kwargs)
File "/opt/conda-analytics/lib/python3.10/site-packages/conda/cli/main_install.py", line 26, in execute
install(args, parser, "install")
File "/opt/conda-analytics/lib/python3.10/site-packages/conda/cli/install.py", line 316, in install
unlink_link_transaction = solver.solve_for_transaction(
File "/opt/conda-analytics/lib/python3.10/site-packages/conda/core/solve.py", line 154, in solve_for_transaction
unlink_precs, link_precs = self.solve_for_diff(
File "/opt/conda-analytics/lib/python3.10/site-packages/conda/core/solve.py", line 215, in solve_for_diff
final_precs = self.solve_final_state(
File "/opt/conda-analytics/lib/python3.10/site-packages/conda_libmamba_solver/solver.py", line 199, in solve_final_state
index = IndexHelper(
File "/opt/conda-analytics/lib/python3.10/site-packages/conda_libmamba_solver/index.py", line 125, in __init__
self._index = self._load_channels()
File "/opt/conda-analytics/lib/python3.10/site-packages/conda_libmamba_solver/index.py", line 281, in _load_channels
info = self._json_path_to_repo_info(url, jsons[url])
File "/opt/conda-analytics/lib/python3.10/site-packages/conda_libmamba_solver/index.py", line 238, in _json_path_to_repo_info
repo = api.Repo(self._pool, noauth_url, str(json_path), escape_channel_url(noauth_url))
RuntimeError: No such file or directory
`$ /opt/conda-analytics/condabin/conda install --yes --offline --freeze-installed --channel local --name 2023-09-22T14.13.19_btullis conda=23.7.4 conda-libmamba-solver=23.7.0`
environment variables:
CIO_TEST=<not set>
CONDA_ROOT=/opt/conda-analytics
CURL_CA_BUNDLE=<not set>
LD_PRELOAD=<not set>
NO_PROXY=<set>
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/home/btullis
/bin
PYTHONPATH=:/srv/deployment/analytics/refinery/python
REQUESTS_CA_BUNDLE=<not set>
SSL_CERT_FILE=<not set>
no_proxy=<set>
active environment : None
user config file : /home/btullis/.condarc
populated config files : /opt/conda-analytics/condarc
/home/btullis/.conda/envs/2023-09-22T14.13.19_btullis/condarc
conda version : 23.7.4
conda-build version : not installed
python version : 3.10.12.final.0
virtual packages : __archspec=1=x86_64
__glibc=2.31=0
__linux=5.10.0=0
__unix=0=0
base environment : /opt/conda-analytics (read only)
conda av data dir : /opt/conda-analytics/etc/conda
conda av metadata url : None
channel URLs : https://conda.anaconda.org/conda-forge/linux-64 (offline)
https://conda.anaconda.org/conda-forge/noarch (offline)
https://repo.anaconda.com/pkgs/main/linux-64 (offline)
https://repo.anaconda.com/pkgs/main/noarch (offline)
https://repo.anaconda.com/pkgs/r/linux-64 (offline)
https://repo.anaconda.com/pkgs/r/noarch (offline)
package cache : /home/btullis/.conda/pkgs
/home/btullis/.conda/envs/2023-09-22T14.13.19_btullis/pkgs
/opt/conda-analytics/pkgs
envs directories : /home/btullis/.conda/envs
/opt/conda-analytics/envs
platform : linux-64
user-agent : conda/23.7.4 requests/2.31.0 CPython/3.10.12 Linux/5.10.0-23-amd64 debian/11 glibc/2.31 solver/libmamba conda-libmamba-solver/23.7.0 libmambapy/1.5.1
UID:GID : 32741:500
netrc file : None
offline mode : True
An unexpected error has occurred. Conda has prepared the above report.
If you suspect this error is being caused by a malfunctioning plugin,
consider using the --no-plugins option to turn off plugins.
Example: conda --no-plugins install <package>
Alternatively, you can set the CONDA_NO_PLUGINS environment variable on
the command line to run the command without plugins enabled.
Example: CONDA_NO_PLUGINS=true conda install <package>
Upload did not complete.Great! I've now got the conda environment cloning to work as well.
It required the use of the classic solver to install conda and the libmamba solver into the cloned environment, before they can then be used by default.
btullis@an-test-client1002:~$ sudo dpkg -i conda-analytics-0.0.23.dev_amd64.deb
(Reading database ... 321857 files and directories currently installed.)
Preparing to unpack conda-analytics-0.0.23.dev_amd64.deb ...
Unpacking conda-analytics (0.0.23) over (0.0.23) ...
Setting up conda-analytics (0.0.23) ...
Post install script.
Running /opt/conda-analytics/bin/python /opt/conda-analytics/bin/conda-unpack...
btullis@an-test-client1002:~$ conda-analytics-list
# conda environments:
#
2023-09-13T16.26.33_btullis /home/btullis/.conda/envs/2023-09-13T16.26.33_btullis
2023-09-14T12.25.12_btullis /home/btullis/.conda/envs/2023-09-14T12.25.12_btullis
2023-09-14T12.28.09_btullis /home/btullis/.conda/envs/2023-09-14T12.28.09_btullis
2023-09-14T12.35.54_btullis /home/btullis/.conda/envs/2023-09-14T12.35.54_btullis
2023-09-15T14.10.17_btullis /home/btullis/.conda/envs/2023-09-15T14.10.17_btullis
2023-09-21T16.18.06_btullis /home/btullis/.conda/envs/2023-09-21T16.18.06_btullis
2023-09-22T14.13.19_btullis /home/btullis/.conda/envs/2023-09-22T14.13.19_btullis
2023-09-22T14.49.14_btullis /home/btullis/.conda/envs/2023-09-22T14.49.14_btullis
btullis-test /home/btullis/.conda/envs/btullis-test
btullis-test2 /home/btullis/.conda/envs/btullis-test2
base /opt/conda-analytics
btullis@an-test-client1002:~$ conda-analytics-clone
Creating new cloned conda env 2023-09-22T16.32.22_btullis...
Source: /opt/conda-analytics
Destination: /home/btullis/.conda/envs/2023-09-22T16.32.22_btullis
The following packages cannot be cloned out of the root environment:
- conda-forge/linux-64::conda-23.7.4-py310hff52083_0
- conda-forge/noarch::conda-libmamba-solver-23.7.0-pyhd8ed1ab_0
Packages: 213
Files: 963
Downloading and Extracting Packages
Downloading and Extracting Packages
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate 2023-09-22T16.32.22_btullis
#
# To deactivate an active environment, use
#
# $ conda deactivate
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/btullis/.conda/envs/2023-09-22T16.32.22_btullis
added / updated specs:
- conda-libmamba-solver=23.7.0
- conda=23.7.4
The following NEW packages will be INSTALLED:
conda conda-forge/linux-64::conda-23.7.4-py310hff52083_0
conda-libmamba-so~ conda-forge/noarch::conda-libmamba-solver-23.7.0-pyhd8ed1ab_0
Downloading and Extracting Packages
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Fri 22 Sep 2023 04:33:35 PM UTC Created user conda environment 2023-09-22T16.32.22_btullis
To activate this environment with vanilla conda run:
source /opt/conda-analytics/etc/profile.d/conda.sh
conda activate 2023-09-22T16.32.22_btullis
Alternatively, you can use the conda-analytic helper script:
source conda-analytics-activate 2023-09-22T16.32.22_btullisNext I will test that jupyterhub and pyspark work.
JupyterHub spawns a new environment correctly and reports the correct libmamba version.
I've got a new merge request here for a 0.0.23 release: https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/35
If anyone feels like testing anything else on an-test-client1002 in the meantime, be my guests.
If this goes well we can hopefully try pushing out this version to production early next week.
btullis merged https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/merge_requests/35
Fix problems with the libmamaba solver
Mentioned in SAL (#wikimedia-analytics) [2023-09-26T10:34:29Z] <btullis> deploying conda-analytics v0.0.23 to hadoop-all for T337258
Mentioned in SAL (#wikimedia-analytics) [2023-09-26T10:36:56Z] <btullis> deploying conda-analytics v0.0.23 to analytics-airflow for T337258
Mentioned in SAL (#wikimedia-analytics) [2023-09-26T10:43:29Z] <btullis> deploying conda-analytics v0.0.23 to stats servers for T337258
I'm seeing some warnings like this when installing packages:
warning libmamba Problem type not implemented SOLVER_RULE_STRICT_REPO_PRIORITY
I suspect that it is related to this entry in the condarc file: https://gitlab.wikimedia.org/repos/data-engineering/conda-analytics/-/blob/main/debian/config/condarc#L16-18
channel_priority: strict
This might need updating to get rid of this configuration. I will check.
I'm seeing strange behaviour from stat1009.
When I try to clone the environment on this host, it fails at the stage of installing conda and libmamba to the new environment.
"${CONDA_BASE_ENV_PREFIX}"/condabin/conda install --yes --offline --freeze-installed --name "${CONDA_NEW_ENV_NAME}" --solver classic conda=23.7.4 conda-libmamba-solver=23.7.0The error output is long so I have added it as a phaste here: P52636
The head of the error message says the following:
btullis@stat1009:~$ conda-analytics-clone T343823-test Creating new cloned conda env T343823-test... Source: /opt/conda-analytics Destination: /home/btullis/.conda/envs/T343823-test The following packages cannot be cloned out of the root environment: - conda-forge/linux-64::conda-23.7.4-py310hff52083_0 - conda-forge/noarch::conda-libmamba-solver-23.7.0-pyhd8ed1ab_0 Packages: 214 Files: 961 Downloading and Extracting Packages Downloading and Extracting Packages Preparing transaction: done Verifying transaction: done Executing transaction: done # # To activate this environment, use # # $ conda activate T343823-test # # To deactivate an active environment, use # # $ conda deactivate Collecting package metadata (current_repodata.json): done Solving environment: unsuccessful initial attempt using frozen solve. Retrying with flexible solve. Collecting package metadata (repodata.json): done Solving environment: / Found conflicts! Looking for incompatible packages. This can take several minutes. Press CTRL-C to abort. failed UnsatisfiableError: The following specifications were found to be incompatible with a past explicit spec that is not an explicit spec in this operation (ld_impl_linux-64):
I'm checking on all of theother stats servers at the moment. stat1009 is the only active bullseye stat server we have at the moment, but an-test-client1002 doesn't exhibit this problem.
btullis@an-test-client1002:~$ conda-analytics-clone T343823-test
Creating new cloned conda env T343823-test...
Source: /opt/conda-analytics
Destination: /home/btullis/.conda/envs/T343823-test
The following packages cannot be cloned out of the root environment:
- conda-forge/linux-64::conda-23.7.4-py310hff52083_0
- conda-forge/noarch::conda-libmamba-solver-23.7.0-pyhd8ed1ab_0
Packages: 214
Files: 963
Downloading and Extracting Packages
Downloading and Extracting Packages
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate T343823-test
#
# To deactivate an active environment, use
#
# $ conda deactivate
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/btullis/.conda/envs/T343823-test
added / updated specs:
- conda-libmamba-solver=23.7.0
- conda=23.7.4
The following NEW packages will be INSTALLED:
conda conda-forge/linux-64::conda-23.7.4-py310hff52083_0
conda-libmamba-so~ conda-forge/noarch::conda-libmamba-solver-23.7.0-pyhd8ed1ab_0
Downloading and Extracting Packages
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Tue 26 Sep 2023 12:57:48 PM UTC Created user conda environment T343823-test
To activate this environment with vanilla conda run:
source /opt/conda-analytics/etc/profile.d/conda.sh
conda activate T343823-test
Alternatively, you can use the conda-analytic helper script:
source conda-analytics-activate T343823-test
btullis@an-test-client1002:~$ cat /etc/debian_version
11.7
btullis@an-test-client1002:~$ apt-cache policy conda-analytics
conda-analytics:
Installed: 0.0.23
Candidate: 0.0.23
Version table:
*** 0.0.23 1001
1001 http://apt.wikimedia.org/wikimedia bullseye-wikimedia/main amd64 Packages
100 /var/lib/dpkg/statusI have removed all previous conda envinronments on stat1009 and now it's working again. I was able to clone a new environment successfully.
btullis@stat1009:~$ rm -rf .conda/
btullis@stat1009:~$ conda-analytics-clone T343823-test
Creating new cloned conda env T343823-test...
Source: /opt/conda-analytics
Destination: /home/btullis/.conda/envs/T343823-test
The following packages cannot be cloned out of the root environment:
- conda-forge/linux-64::conda-23.7.4-py310hff52083_0
- conda-forge/noarch::conda-libmamba-solver-23.7.0-pyhd8ed1ab_0
Packages: 214
Files: 961
Downloading and Extracting Packages
Downloading and Extracting Packages
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate T343823-test
#
# To deactivate an active environment, use
#
# $ conda deactivate
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/btullis/.conda/envs/T343823-test
added / updated specs:
- conda-libmamba-solver=23.7.0
- conda=23.7.4
The following NEW packages will be INSTALLED:
conda conda-forge/linux-64::conda-23.7.4-py310hff52083_0
conda-libmamba-so~ conda-forge/noarch::conda-libmamba-solver-23.7.0-pyhd8ed1ab_0
Downloading and Extracting Packages
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Thu 28 Sep 2023 04:33:21 PM UTC Created user conda environment T343823-test
To activate this environment with vanilla conda run:
source /opt/conda-analytics/etc/profile.d/conda.sh
conda activate T343823-test
Alternatively, you can use the conda-analytic helper script:
source conda-analytics-activate T343823-test
btullis@stat1009:~$Marking this ticket as resolved.