Page MenuHomePhabricator

Add Java 11 Image to Config Docker Images
Closed, ResolvedPublic

Description

Sonar cloud will no longer run with Java 8 in October 2020, details here: https://sonarcloud.io/documentation/appendices/end-of-support/

We need to update the Sonar to run in Java 11 but there is no Java 11 image currently in the Config images.

Notes from IRC:

  • The ci images comes from releng/java8 which comes from releng/ci-stretch
  • There is no Java 11 available for stretch, so we need to bump up to buster
  • We should rename the Java 8 CI image to Maven (suggested by @JAllemandou)
    • And rename the java8-sonar-scanner to sonar-scanner and it should inherit from releng/maven from above

To support each of Java 8 and Java 11 we can either:
)

  • Have a single Buster image that has Java 11 (from Debian) and Java8 (from component/jdk8). The JAVA_HOME env variable can be used to switch between Java versions.
    • One would need to specific the proper JAVA_HOME somehow.
    • Cause the images to be rather large (two jdk installed)
  • A common maven image that has sh entry point and then child images for each of java 11 and java 8
    • A user or the CI job then decide which java version to use based on the image name (example: maven-jdk8, maven-jdk11)

Event Timeline

Mstyles created this task.Aug 4 2020, 5:25 PM
Restricted Application added a subscriber: Aklapper. ยท View Herald TranscriptAug 4 2020, 5:25 PM
hashar updated the task description. (Show Details)Aug 4 2020, 6:29 PM

For Java 11 we have to base the releng/java8-sonar-scanner to be on Buster and we should also rename it to drop the java prefix from the image name.

While at it, I would to take the opportunity to add java 11 in the CI images. I am probably in favor of having two separate images (maven-jdk8 and maven-jdk11). That seems simpler than having to mess up with JAVA_HOME to pick a different java version than the default one.

hashar triaged this task as Medium priority.Aug 4 2020, 6:31 PM
Gehel added a comment.Thu, Sep 24, 2:39 PM

Note that for Java projects, we want to continue to do the main build under Java 8, but the analysis with Java 11.

@thcipriani and @greg this is the task that prompted the SonarCloud product ownership discussion and subsequent email to y'all.

hashar added a comment.Tue, Oct 6, 4:42 PM

@zeljkofilipin I am on train duty this week. I can probably just do it, but I guess we can pair it next week? So at least you will be slightly familiar with how we build the Docker image which might prove helpful in the long run :]

@hashar Yes, please! Let me know when ever you're ready to work on this. I would be more than glad to see how it's done.

hashar claimed this task.Mon, Oct 12, 11:58 AM

I will refactor the images in order to share material between the java8 and java11 version and add a layer of images with just java 8 and java 11.

Change 633523 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] dockerfiles: java 11 image

https://gerrit.wikimedia.org/r/633523

Change 633524 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] dockerfiles: use a scratch image for maven

https://gerrit.wikimedia.org/r/633524

Change 633523 merged by jenkins-bot:
[integration/config@master] dockerfiles: java 11 image

https://gerrit.wikimedia.org/r/633523

Change 633531 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] dockerfiles: "move" java8 to maven-java8

https://gerrit.wikimedia.org/r/633531

Change 633532 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] dockerfiles: maven-java8 and maven-java11

https://gerrit.wikimedia.org/r/633532

Change 633531 merged by jenkins-bot:
[integration/config@master] dockerfiles: "move" java8 to maven-java8

https://gerrit.wikimedia.org/r/633531

Change 633524 merged by jenkins-bot:
[integration/config@master] dockerfiles: use a scratch image for maven

https://gerrit.wikimedia.org/r/633524

Change 633532 merged by jenkins-bot:
[integration/config@master] dockerfiles: maven-java8 and maven-java11

https://gerrit.wikimedia.org/r/633532

Mentioned in SAL (#wikimedia-releng) [2020-10-12T13:58:56Z] <hashar> Successfully tagged docker-registry.discovery.wmnet/releng/maven-java11:1.0.0 # T259646

Change 633549 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] dockerfiles: change base image of java8-* images

https://gerrit.wikimedia.org/r/633549

Change 633549 merged by jenkins-bot:
[integration/config@master] dockerfiles: change base image of java8-* images

https://gerrit.wikimedia.org/r/633549

Change 633554 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] dockerfiles: rename sonar-scanner and use java11

https://gerrit.wikimedia.org/r/633554

Change 633554 merged by jenkins-bot:
[integration/config@master] dockerfiles: rename sonar-scanner and use java11

https://gerrit.wikimedia.org/r/633554

Change 633558 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] jjb: update codehealth jobs to Java11

https://gerrit.wikimedia.org/r/633558

Mentioned in SAL (#wikimedia-releng) [2020-10-12T15:22:29Z] <hashar> Updated non voting codehealth jobs to Java11 for T259646: mwcore-codehealth-master-non-voting mwext-codehealth-master-non-voting

I have updated the two non voting jobs to use the Java 11 based Docker images:

https://integration.wikimedia.org/ci/job/mwcore-codehealth-master-non-voting
https://integration.wikimedia.org/ci/job/mwext-codehealth-master-non-voting

If that looks ok we can do the rest I guess.

I have updated the two non voting jobs to use the Java 11 based Docker images:

https://integration.wikimedia.org/ci/job/mwcore-codehealth-master-non-voting
https://integration.wikimedia.org/ci/job/mwext-codehealth-master-non-voting

If that looks ok we can do the rest I guess.

๐Ÿ‘ thanks!

From a successful build of mwcore-codehealth-master-non-voting:

INFO: SonarQube Scanner 4.0.0.1744
INFO: Java 11.0.8 Debian (64-bit)

Change 633558 merged by jenkins-bot:
[integration/config@master] jjb: update codehealth jobs to Java11

https://gerrit.wikimedia.org/r/633558

hashar closed this task as Resolved.Thu, Oct 15, 8:07 AM

In order to support Java 11 in CI Docker image, I had to do a lot of various refactoring. When previously we had:

ci-stretch
 \-- java8
    \-- java8-sonar-scanner

We now have a hierarchy that looks like:

ci-stretch
 \-- java8
    \-- maven-java8
ci-buster
  \-- java11
    \-- maven-java11
      \-- sonar-scanner

And the actual change to fulfill that task ends up being straightforward: just change the parent image from maven-java8 to maven-java11 and drop the custom NodeJS that was required for Debian Stretch. https://gerrit.wikimedia.org/r/c/integration/config/+/633554/2/dockerfiles/sonar-scanner/Dockerfile.template

--- a/dockerfiles/java8-sonar-scanner/Dockerfile.template
+++ b/dockerfiles/sonar-scanner/Dockerfile.template
@@ -1,12 +1,11 @@
-FROM {{ "maven-java8" | image_tag }}
+FROM {{ "maven-java11" | image_tag }}
 
 USER root
 COPY KEYS /tmp/KEYS
 
 ENV SONAR_SCANNER_VERSION=4.0.0.1744
 
-RUN echo "deb http://apt.wikimedia.org/wikimedia stretch-wikimedia component/node10" > /etc/apt/sources.list.d/stretch-node10.list \
-    && {{ "nodejs gnupg wget unzip curl jq" | apt_install }} \
+RUN {{ "nodejs gnupg wget unzip curl jq" | apt_install }} \
     && git clone --depth 1 https://gerrit.wikimedia.org/r/p/integration/npm.git /srv/npm \
     && rm -rf /srv/npm/.git \
     && ln -s /srv/npm/bin/npm-cli.js /usr/bin/npm \