Page MenuHomePhabricator

NEW BUG REPORT conda-analytics-clone creates environments named with a trailing brace
Closed, ResolvedPublicBUG REPORT

Description

What kind of problem are you reporting?

  • Access related problem
  • Service related problem
  • Data related problem

When you run conda-analytics-clone to create a new environment, the name of the environment environment now includes a trailing brace. For example, 2024-07-03T18.21.50_nshahquinn-wmf} instead of 2024-07-03T18.21.50_nshahquinn-wmf.

This is due to a typo in line 11 of conda-analytics-clone:

DEFAULT_CONDA_NEW_ENV_NAME="$(date '+%Y-%m-%dT%H.%M.%S')_${USER}}"
#                                                               ^ extra brace

In most cases, this does not cause problems, but it seems to cause development installs of Python packages to fail. Here's an example: P65767. The key line is:

 File "/home/nshahquinn-wmf/.conda/envs/2024-07-03T19.55.05_nshahquinn-wmf}/lib/python3.10/site-packages/setuptools/_distutils/util.py", line 212, in subst_vars
    return _subst_compat(s).format_map(lookup)
ValueError: Single '}' encountered in format string

Event Timeline

nshahquinn-wmf created this task.
nshahquinn-wmf raised the priority of this task from Low to High.Jul 3 2024, 10:23 PM
nshahquinn-wmf renamed this task from NEW BUG REPORT conda-analytics-clone creates environments with a trailing brace to NEW BUG REPORT conda-analytics-clone creates environments named with a trailing brace.Jul 3 2024, 10:30 PM
nshahquinn-wmf lowered the priority of this task from High to Medium.

I'm testing this now on an-test-client1002.

btullis@an-test-client1002:~$ wget https://gitlab.wikimedia.org/api/v4/projects/359/packages/generic/conda-analytics/0.0.35/conda-analytics-0.0.35_amd64.deb
--2024-07-04 16:31:52--  https://gitlab.wikimedia.org/api/v4/projects/359/packages/generic/conda-analytics/0.0.35/conda-analytics-0.0.35_amd64.deb
Resolving gitlab.wikimedia.org (gitlab.wikimedia.org)... 2620:0:860:1:208:80:153:8, 208.80.153.8
Connecting to gitlab.wikimedia.org (gitlab.wikimedia.org)|2620:0:860:1:208:80:153:8|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1080788932 (1.0G) [application/octet-stream]
Saving to: ‘conda-analytics-0.0.35_amd64.deb’

conda-analytics-0.0.35_amd64.deb                     100%[=====================================================================================================================>]   1.01G  79.8MB/s    in 11s     

2024-07-04 16:32:04 (90.3 MB/s) - ‘conda-analytics-0.0.35_amd64.deb’ saved [1080788932/1080788932]

btullis@an-test-client1002:~$ sudo dpkg -i conda-analytics-0.0.35_amd64.deb 
(Reading database ... 258857 files and directories currently installed.)
Preparing to unpack conda-analytics-0.0.35_amd64.deb ...
Unpacking conda-analytics (0.0.35) over (0.0.32) ...
Setting up conda-analytics (0.0.35) ...
Post install script.
  Running /opt/conda-analytics/bin/python /opt/conda-analytics/bin/conda-unpack...

I'll verify that it can do:

  • conda-analytics-clone mycoolenv
  • conda-analytics-clone (without any arguments)
  • That the jupyterhub-conda.service starts and functions with the new base environment
  • That starting a new jupyterhub session with a newly cloned environment works as expected

This looks good.

btullis@an-test-client1002:~$ cat .conda/envs/mycoolenv/conda-meta/pinned
pyspark=3.1.2
numpy<1.24.0
pandas<2.0.0
pyarrow=9.0.0
jupyter_core=5.5
jupyterhub=1.5.0
jupyterhub-systemdspawner=0.15.0
jupyterhub-ldapauthenticator=1.3.2
jupyterlab_server=2.25
sqlalchemy<2.0
jupyterlab=3.4.8

So does this:

image.png (884×1 px, 93 KB)

I'm happy to roll this out, but I'll probably wait until Monday, given that it also includes the production pyspark enviroment.

Mentioned in SAL (#wikimedia-analytics) [2024-07-08T10:21:05Z] <btullis> rolling out conda-analytics version 0.0.35 for T369240 and T369210