Page MenuHomePhabricator

Rebuild hive-hcatalog package for bullseye to address missing symlinks
Open, Needs TriagePublic


We have upgraded one of our Hadoop workers to bullseye, but have discovered a problem with one of the packages.

This is the hive-hcatalog packages, which is missing a vital set of symlinks in the /usr/lib/hive-hcatalog/share/hcatalog/ directory.

That directory on a buster host contains this:

btullis@an-test-worker1002:~$ ls -l /usr/lib/hive-hcatalog/share/hcatalog/
total 516
-rw-r--r-- 1 root root 264740 Jan  4  2022 hive-hcatalog-core-2.3.6.jar
lrwxrwxrwx 1 root root     28 Jan  4  2022 hive-hcatalog-core.jar -> hive-hcatalog-core-2.3.6.jar
-rw-r--r-- 1 root root  53963 Jan  4  2022 hive-hcatalog-pig-adapter-2.3.6.jar
lrwxrwxrwx 1 root root     35 Jan  4  2022 hive-hcatalog-pig-adapter.jar -> hive-hcatalog-pig-adapter-2.3.6.jar
-rw-r--r-- 1 root root  73711 Jan  4  2022 hive-hcatalog-server-extensions-2.3.6.jar
lrwxrwxrwx 1 root root     41 Jan  4  2022 hive-hcatalog-server-extensions.jar -> hive-hcatalog-server-extensions-2.3.6.jar
-rw-r--r-- 1 root root 128401 Jan  4  2022 hive-hcatalog-streaming-2.3.6.jar
lrwxrwxrwx 1 root root     33 Jan  4  2022 hive-hcatalog-streaming.jar -> hive-hcatalog-streaming-2.3.6.jar

On the bullseye host, those unversioned symlinks are missing:

btullis@an-test-worker1001:~$ ls -l /usr/lib/hive-hcatalog/share/hcatalog/
total 520
-rw-r--r-- 1 root root 264798 Aug 12  2022 hive-hcatalog-core-2.3.6.jar
-rw-r--r-- 1 root root  54023 Aug 12  2022 hive-hcatalog-pig-adapter-2.3.6.jar
-rw-r--r-- 1 root root  73772 Aug 12  2022 hive-hcatalog-server-extensions-2.3.6.jar
-rw-r--r-- 1 root root 128459 Aug 12  2022 hive-hcatalog-streaming-2.3.6.jar

The script contains a section that was supposed to create those symlinks at the time of the package creation:

    (cd $DIR &&
     for j in hive-hcatalog-*.jar; do
       if [[ $j =~ hive-hcatalog-(.*)-${HIVE_VERSION}.jar ]]; then
         ln -s $j hive-hcatalog-$name.jar

We need to understand why that script didn't work as expected and rebuild the package.
At the same time as fixing this issue, we need to:

  1. Be on the lookout for any other implications or occurrences of this issue about missing symlinks.
  2. Update the instructions on Wikitech for building our hadoop packages - Currently the best instructions are in T310643: Build Bigtop 1.5 Hadoop packages for Bullseye
  3. Refine the build process if necessary/possible