Page MenuHomePhabricator

Weird dependency issue on the stretch grid around libmariadb-dev-compat and friends
Closed, ResolvedPublic


So something is happening on the stretch grid that has baffled me for some time, but it has broken puppet and apt repeatedly on tools-sgebastion-06.

libgdal-dev (a gis, mappy kinda library) is installed because someone needed it at some point on the old grid. I presume they will still require it on the new grid. So that depends on libmysqlclient-dev which actually installs libmariadbclient-dev-compat (which just creates a few symlinks to convince things they can link against mysql libs). We also have libmariadb-dev-compat which creates the same damned symlinks and a few more.

The problem is that those two packages (libmariadbclient-dev-compat and libmariadb-dev-compat) are mutually exclusive packages. If I could choose one, I'd choose libmariadb-dev-compat because it is a superset of the other and might be better for developers as a result. Sadly puppet is reinstalling things back and forth it seems. On the bastion, it has begun dying on install. You end up with broken apt, and you have to run dpkgs --configure -a to get it working again. Then it breaks on the next run. It's really fun.

To make this story all the better, in Debian Sid, libmariadbclient-dev-compat is a virtual package that points to libmariadb-dev-compat ❤️. This seems like what it should be everywhere, but it is not. Also, a backport would mean our mariadb packages are at 10.3 rather than 10.1 like the actual databases are. I think the solution might be to remove libmariadb-dev-compat from modules/profile/manifests/toolforge/grid/exec_environ.pp even though it seems like the better one, but I wanted to create this ticket to get help from @aborrero to make sure.

Event Timeline

Bstorm triaged this task as Medium priority.Feb 8 2019, 12:46 AM
Bstorm created this task.

I have realized that the worst of this issue (the crashing and requiring dpkg --configure -a) may be because of the cgroup limits. Puppet is getting SIGKILLed during apt runs because I'm running it using sudo. The rest is probably just a matter of removing libmariadb-dev-compat.

Noticed the problems around cgroups here: T215434 and have greatly expanded the limits for now.

Yes! That was it. It just dances back and forth installing one dependency or the other on each puppet run like it always did now :)

TL;DR: I think we can have this combo installed: libgdal-dev libmariadbclient-dev-compat libmariadbclient-dev (even though I have doubts we need the -compat one). Will create a patch proposal for this.

From a quick reading, my understanding is that the '-compat' package provides some sort of compatibility with mysql, specifically to allow co-existance of mysql and mariadb client libraries in the system.
Since you mentioned the database is mariadb, perhaps we can forget about this compatibility?

To make things even more confusing, there are 2 flavors of the -dev packages, identified by the client keyword. Apparently they have different licensing:

  • libmariadb-dev: LGPL
  • libmariadbclient-dev: GPL
root@tools-sgebastion-06:~# aptitude show libmariadb-dev
Package: libmariadb-dev           
Version: 2.3.2-2
The main reason to use this package <<libmariadb-dev>>, rather than libmariadbclient-dev, is differences in licensing: this package uses the LGPL, while libmariadbclient-dev uses the GPL.

And we also have 2 flavors for the -compat version:

  • libmariadb-dev-compat: LGPL, allows co-existence with mysql libs
  • libmariadbclient-dev-compat: GPL, allows co-existence with mysql libs

I just did a quick installation test to see how the resolver behaves in this situation:

test1 -- NOOK
root@tools-sgebastion-06:~# apt-get install libgdal-dev libmariadb-dev-compat libmariadb-dev -sy
Reading package lists... Done
Building dependency tree       
Reading state information... Done
libmariadb-dev is already the newest version (2.3.2-2).
libmariadb-dev-compat is already the newest version (2.3.2-2).
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 libgdal-dev : Depends: default-libmysqlclient-dev but it is not going to be installed
 libmariadb-dev-compat : Conflicts: libmariadbclient-dev but 10.1.37-0+deb9u1 is to be installed
E: Unable to correct problems, you have held broken packages.
test2 -- OK
root@tools-sgebastion-06:~# apt-get install libgdal-dev libmariadbclient-dev-compat libmariadbclient-dev -sy
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
Suggested packages:
The following packages will be REMOVED:
The following NEW packages will be installed:
  default-libmysqlclient-dev libgdal-dev libmariadbclient-dev libmariadbclient-dev-compat
0 upgraded, 4 newly installed, 1 to remove and 22 not upgraded.
Remv libmariadb-dev-compat [2.3.2-2]
Inst libmariadbclient-dev (10.1.37-0+deb9u1 Debian-Security:9/stable [amd64])
Inst libmariadbclient-dev-compat (10.1.37-0+deb9u1 Debian-Security:9/stable [amd64])
Inst default-libmysqlclient-dev (1.0.2 Debian:9.7/stable [amd64])
Inst libgdal-dev (2.1.2+dfsg-5 Debian:9.7/stable [amd64])
Conf libmariadbclient-dev (10.1.37-0+deb9u1 Debian-Security:9/stable [amd64])
Conf libmariadbclient-dev-compat (10.1.37-0+deb9u1 Debian-Security:9/stable [amd64])
Conf default-libmysqlclient-dev (1.0.2 Debian:9.7/stable [amd64])
Conf libgdal-dev (2.1.2+dfsg-5 Debian:9.7/stable [amd64])

So this combination seems right: libgdal-dev libmariadbclient-dev-compat libmariadbclient-dev, which only introduce the licensing change from what we have in puppet right now.

Change 489195 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] toolforge: grid: exec_environ: use libmariadbclient* packages

Change 489195 merged by GTirloni:
[operations/puppet@production] toolforge: grid: exec_environ: use libmariadbclient-dev* packages

Closing task. Feel free to reopen if required.