Page MenuHomePhabricator

Install Java 7 in Debian Stretch (to fix sqoop (and possibly other things) on stat1005)
Closed, DeclinedPublic

Description

At https://phabricator.wikimedia.org/T170052#3450141, @GoranSMilovanovic noted that sqoop fails on stat1005, while it works fine on stat1002. His failed job contains this error message:

org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.UnsupportedClassVersionError: QueryResult : Unsupported major.minor version 52.0

We've seen this error before: when we tried to use Druid with Java 8, while the Hadoop cluster runs Java 7.

We installed stat1005 as Debian Stretch, so that it could have access to newer drivers compatible with its special GPU. However, openjdk-7 is not available in Debian Stretch. It is available in Jessie.

Can we just forward port openjdk-7 (maybe copy the .deb?) over from Jessie?

Event Timeline

Yar, I think this is not going to work.

I tried on a Stretch labs instance. There a few (innocuous) dependencies that also aren't in stretch (tzdata-java), that would conflict if we tried to bring them forward too.

Really not sure what to do here. Perhaps we can keep Stretch and Java 8 for stat1005, and keep stat1002 around until we upgrade the whole Hadoop cluster to Java 8.

We should really avoid a one off openjdk-7 build for stretch, these really need to kept up-to-date with every quartely Oracle CPU. We already had the situation where a similar outdated openjdk-8 build for trusty (which was originally imported from a PPA for tool labs) ended up on our production elastic cluster (before it was reimaged to jessie). Java has regular high severity security issues, it's not a piece of infrastructure where we can be lax. And continously rebuilding openjdk-7 for a single host isn't particularly appealing either.

My recommendation would to run those components which are tied to our Hadoop cluster (and thus tied to the Java version support by CDH) on one of the stat100[23] boxes, ideally reimaged to jessie so that they use the same base OS as the Hadoop cluster. And when the analytics* hosts are upgraded, simply decomission the box for good.

My recommendation would to run those components which are tied to our Hadoop cluster (and thus tied to the Java version support by CDH) on one of the stat100[23] boxes, ideally reimaged to jessie so that they use the same base OS as the Hadoop cluster. And when the analytics* hosts are upgraded, simply decomission the box for good.

Completely agree, this is what we have decided yesterday (me and Andrew) after a chat about the next steps. We'll keep stat1002 until the Hadoop cluster is migrated to java8 :)

+1.

You know @elukey..DUH stat1004 is Jessie and is a Hadoop client and has Java 7. We don't need to keep stat1002. Dunno why we didn't think of that yesterday.

Declining this ticket.

Ah right! Even better then! We focused only on the brand new nodes, forgot the old glories :)