So something is happening on the stretch grid that has baffled me for some time, but it has broken puppet and apt repeatedly on tools-sgebastion-06.
libgdal-dev (a gis, mappy kinda library) is installed because someone needed it at some point on the old grid. I presume they will still require it on the new grid. So that depends on libmysqlclient-dev which actually installs libmariadbclient-dev-compat (which just creates a few symlinks to convince things they can link against mysql libs). We also have libmariadb-dev-compat which creates the same damned symlinks and a few more.
The problem is that those two packages (libmariadbclient-dev-compat and libmariadb-dev-compat) are mutually exclusive packages. If I could choose one, I'd choose libmariadb-dev-compat because it is a superset of the other and might be better for developers as a result. Sadly puppet is reinstalling things back and forth it seems. On the bastion, it has begun dying on install. You end up with broken apt, and you have to run dpkgs --configure -a to get it working again. Then it breaks on the next run. It's really fun.
To make this story all the better, in Debian Sid, libmariadbclient-dev-compat is a virtual package that points to libmariadb-dev-compat ❤️. This seems like what it should be everywhere, but it is not. Also, a backport would mean our mariadb packages are at 10.3 rather than 10.1 like the actual databases are. I think the solution might be to remove libmariadb-dev-compat from modules/profile/manifests/toolforge/grid/exec_environ.pp even though it seems like the better one, but I wanted to create this ticket to get help from @aborrero to make sure.