Page MenuHomePhabricator

libmariadbclient.so.18 is missing from tcl type kubernetes webservice
Closed, ResolvedPublic

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 25 2019, 4:26 AM
bd808 added a subscriber: bd808.Jan 25 2019, 6:08 AM

Possibly related to the rebuild of the Kubernetes containers at 2019-01-22T20:21UTC where the base image switched to use the production jessie base image (rODIT7bd63bff4f95: Use production wikimedia-jessie base image).

bd808 added a comment.Jan 25 2019, 6:20 AM

@Giftpflanze are you sure this is a regression? The tcl Docker container is based on Debian Jessie and has libmysqlclient18 installed.

$ webservice --backend=kubernetes tcl shell
$ dpkg -l | awk '{print $2, $3}' | grep mysql
libmysqlclient18:amd64 5.5.62-0+deb8u1
mysql-common 5.5.62-0+deb8u1
mysqltcl 3.052-1
$ dpkg -L libmysqlclient18
/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libmysqlclient.so.18.0.0
/usr/share
/usr/share/doc
/usr/share/doc/libmysqlclient18
/usr/share/doc/libmysqlclient18/changelog.Debian.gz
/usr/share/doc/libmysqlclient18/changelog.gz
/usr/share/doc/libmysqlclient18/NEWS.Debian.gz
/usr/share/doc/libmysqlclient18/copyright
/usr/lib/x86_64-linux-gnu/libmysqlclient.so.18
/usr/lib/x86_64-linux-gnu/libmysqlclient_r.so.18
/usr/lib/x86_64-linux-gnu/libmysqlclient_r.so.18.0.0

I haven't tried to check older images in the docker repo to see if the switch from the tools specific base image to the production base image changed this somehow, but since the other tickets related to your tools recently were asking for changes in the Stretch grid engine environment I'm wondering if you built your binaries on Stretch and are now trying to run them on Jessie and hitting this problem as a result.

bd808 added a comment.Jan 25 2019, 5:42 PM
[13:20]  <   annika>	bd808: regarding T214668, if the container had libmariadbclient.so.18 and now has libmysqlclient.so.18, how is this not a regression? but you're right, i compiled on the stretch bastion and apparently i run them on jessie

I don't think it is a regression because I downloaded an older version of the tcl container from our Docker registry and found that it too uses libmysql rather than libmariadb:

$ docker run -it docker-registry.tools.wmflabs.org/toollabs-tcl-base:20181004-bkp /bin/bash
$ dpkg -l | awk '{print $2, $3}' | grep mysql
libmysqlclient18:amd64 5.5.60-0+deb8u1
mysql-common 5.5.60-0+deb8u1
mysqltcl 3.052-1

There are 2 ways to fix the issue you are having:

  1. You can rebuild your custom tcl binary on a Debian Jessie system linked against libmysqlclient18
  2. We can change the base image of the tcl Docker image to be docker-registry.tools.wmflabs.org/toollabs-stretch

The second route seems better since there is currently only one webservice using the tcl image:

$ sudo -i kubectl get pods --all-namespaces -o=jsonpath="{.items[*].spec.containers[*].image}" | sed 's/ /\n/g' | grep toollabs | sort | uniq -c | sort -h
      1 docker-registry.tools.wmflabs.org/toollabs-golang-web:latest
      1 docker-registry.tools.wmflabs.org/toollabs-tcl-web:latest
      2 docker-registry.tools.wmflabs.org/toollabs-python2-base:latest
      4 docker-registry.tools.wmflabs.org/toollabs-jdk8-web:latest
      5 docker-registry.tools.wmflabs.org/toollabs-ruby-web:latest
      6 docker-registry.tools.wmflabs.org/toollabs-nodejs-base:latest
      9 docker-registry.tools.wmflabs.org/toollabs-python-base:latest
     16 docker-registry.tools.wmflabs.org/toollabs-nodejs-web:latest
     17 docker-registry.tools.wmflabs.org/toollabs-python2-web:latest
     67 docker-registry.tools.wmflabs.org/toollabs-php72-web:latest
    117 docker-registry.tools.wmflabs.org/toollabs-python-web:latest
    193 docker-registry.tools.wmflabs.org/toollabs-php-web:latest
$ sudo -i kubectl get pods --all-namespaces -o json | jq '.items[]|select(.spec.containers[].image=="docker-registry.tools.wmflabs.org/toollabs-tcl-web:latest")|.metadata.name'
"giftbot-2261944348-ibul4"

I wonder why it worked with the new tclsh binary and tcl libraries on the old tcl container then.

I agree that the base image of the tcl image should be changed to the stretch one.

Change 486539 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/docker-images/toollabs-images@master] tcl: switch base image from jessie to stretch

https://gerrit.wikimedia.org/r/486539

Change 486539 merged by jenkins-bot:
[operations/docker-images/toollabs-images@master] tcl: switch base image from jessie to stretch

https://gerrit.wikimedia.org/r/486539

Change 486546 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/docker-images/toollabs-images@master] tcl/web: Create /var/run/lighttpd

https://gerrit.wikimedia.org/r/486546

Change 486546 merged by jenkins-bot:
[operations/docker-images/toollabs-images@master] tcl/web: Create /var/run/lighttpd

https://gerrit.wikimedia.org/r/486546

Mentioned in SAL (#wikimedia-cloud) [2019-01-25T20:50:03Z] <bd808> Deployed new tcl/web Kubernetes image based on Debian Stretch (T214668)

bd808 closed this task as Resolved.Jan 25 2019, 8:54 PM
bd808 claimed this task.
$ webservice --backend=kubernetes tcl shell
$ dpkg -l | awk '{print $2, $3}' | grep libm
libm17n-0:amd64 1.7.0-3+b1
libmagic-mgc 1:5.30-1+deb9u2
libmagic1:amd64 1:5.30-1+deb9u2
libmagickcore-6.q16-3:amd64 8:6.9.7.4+dfsg-11+deb9u6
libmagickwand-6.q16-3:amd64 8:6.9.7.4+dfsg-11+deb9u6
libmariadbclient18:amd64 10.1.37-0+deb9u1
libmount1:amd64 2.29.2-1+deb9u1
libmpfr4:amd64 3.1.5-1
$ dpkg -L libmariadbclient18
/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libmariadbclient.so.18.0.0
/usr/lib/x86_64-linux-gnu/mariadb18
/usr/lib/x86_64-linux-gnu/mariadb18/plugin
/usr/lib/x86_64-linux-gnu/mariadb18/plugin/client_ed25519.so
/usr/lib/x86_64-linux-gnu/mariadb18/plugin/dialog.so
/usr/lib/x86_64-linux-gnu/mariadb18/plugin/disks.so
/usr/lib/x86_64-linux-gnu/mariadb18/plugin/mysql_clear_password.so
/usr/share
/usr/share/doc
/usr/share/doc/libmariadbclient18
/usr/share/doc/libmariadbclient18/changelog.Debian.gz
/usr/share/doc/libmariadbclient18/copyright
/usr/lib/x86_64-linux-gnu/libmariadbclient.so.18