Page MenuHomePhabricator

Remove Java 8 images from integration/config
Closed, ResolvedPublic

Description

Note: when resolving this task, update the Java section at https://www.mediawiki.org/wiki/Continuous_integration/Language_Versions_Policy


This task is to phase out support for Java 8 from the CI Jenkins jobs triggered by Zuul. They are defined in https://gerrit.wikimedia.org/g/integration/config/

The reason is OpenJDK 8 is not present in Debian Buster (released in July 2019, EOL since August 2022) and we forward ported it. As we would like to phase out Debian Buster images from CI, it looks like we should remove support for Java 8 as a prerequisite.

OpenJDK 11 was introduced in Buster, is in Bullseye and is not in Bookworm (which comes with Java 17).

As an alternative, we can migrate the CI images from BusterBullseye since we forward ported Java 8 there as well (bullseye-wikimedia component/jdk8).

Repositories triggering at least one job having java8 in its name:

Gerrit projectJava 11 jobs?Comment
analytics/gobblin-wmfAdded by Gerrit 974196Fails to build due to a Maven repo not being allowed to fetch eigenbase:eigenbase-properties
analytics/wmde/toolkit-analyzerNoArchive? T351291
integration/gearman-javaYes
mediawiki/tools/mwdumperNoRequested archival with T351228

Done

Repositories that have been investigated with either outcome:

  • Stick to Java 8
  • Java 11 jobs added on top of existing Java 8 jobs
  • CI Migrated from Java 8 to Java 11
  • Project got archived
Gerrit projectJava 11 jobs?Comment
analytics/refinery/sourceNo (doc: Gerrit 973089)No plans for upgrading to Java 11.
integration/jenkinsci/gearman-javaYesArchived project (Gerrit 974205)
operations/software/gerrit/plugins/barricadeNoArchived project (Gerrit 974199
search/cirrus-streaming-updaterYesjavadoc fix for T351413 Gerrit #975805
search/extra-analysisAdded by Gerrit 973083already running on Java 11 in production
search/extraAdded by Gerrit 973083already running on Java 11 in production
search/glentAdded by Gerrit 973083is a spark job so have to remain java8 compatible but can be made java11 ready
search/highlighterAdded by Gerrit 973083already running on Java 11 in production
wikimedia/discovery/discovery-maven-tool-configs811 by Gerrit 974488trivial to migrate
wikimedia/discovery/discovery-parent-pom811 by Gerrit 974488trivial to migrate
wikidata/query/blazegraphNo (doc: Gerrit 973088)We rarely need to build Blazegraph itself. It would be nice to keep a working CI in case we need it, but if needed, we can disable those jobs
wikidata/query/rdfNo (doc: Gerrit 973088)Blazegraph is not running on Java 11 and is unlikely to ever be
wikimedia-event-utilitiesAdded by Gerrit 975800
wmf-jvm-utilsAdded by Gerrit 975352javadoc not found: T351413

Event Timeline

General note: I think we should use the same Java version in CI and in production. While backward compatibility tends to be really good with Java, I think it is best to test our code on the same JVM / JRE than what will ultimately be deployed. We do have a few projects that are not quite ready to be migrated to Java 11 (I'll update some of the comments in the task description).

Should we take this opportunity to migrate those repositories and jobs to Gitlab?

hashar renamed this task from Remove Java 8 to Remove Java 8 images from integration/config.Nov 6 2023, 3:26 PM

Thanks! Given there are a few use cases for keeping Java 8 for now, I will migrate the Docker images from Buster to Bullseye since we have port of Java 8 there (my main intention for this task is to phase out Buster).
While at it, I will add Java 11 jobs to the few repositories that already run Java 11 in production but lack Java 11 CI jobs.

Change 973083 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Add Java 11 jobs to some search/* repositories

https://gerrit.wikimedia.org/r/973083

Change 973088 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] zuul: comment Blazegraph is sticking to Java 8

https://gerrit.wikimedia.org/r/973088

Change 973089 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] zuul: document Refinery is sticking to Java 8

https://gerrit.wikimedia.org/r/973089

Change 973088 merged by jenkins-bot:

[integration/config@master] zuul: comment Blazegraph is sticking to Java 8

https://gerrit.wikimedia.org/r/973088

Change 973083 merged by jenkins-bot:

[integration/config@master] Add Java 11 jobs to some search/* repositories

https://gerrit.wikimedia.org/r/973083

I have send dummy patches for each of the four repositories:

https://gerrit.wikimedia.org/r/c/search/extra/+/973355 and https://gerrit.wikimedia.org/r/c/search/highlighter/+/330661 pass java 11.

https://gerrit.wikimedia.org/r/c/search/glent/+/973357 fails on Java 11, I have filed it as T350974

https://gerrit.wikimedia.org/r/c/search/extra-analysis/+/973356 fails on BOTH Java 8 and Java 11 due to a missing project on SonarQube. I have filed it as T350978

hashar renamed this task from Remove Java 8 images from integration/config to integration/config Remove Java 8 or migrate from Buster to Bullseye.Nov 13 2023, 3:58 PM
hashar renamed this task from integration/config Remove Java 8 or migrate from Buster to Bullseye to Remove Java 8 images from integration/config.

Change 973089 merged by jenkins-bot:

[integration/config@master] zuul: document Refinery is sticking to Java 8

https://gerrit.wikimedia.org/r/973089

Change 974196 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Add Java 11 jobs to analytics/gobblin-wmf

https://gerrit.wikimedia.org/r/974196

Change 974488 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Move maven-tool-config and parent-pom to Java 11

https://gerrit.wikimedia.org/r/974488

Change 974488 merged by jenkins-bot:

[integration/config@master] Move maven-tool-config and parent-pom to Java 11

https://gerrit.wikimedia.org/r/974488

Change 975352 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Add Java 11 to wmf-jvm-utils

https://gerrit.wikimedia.org/r/975352

Change 975353 had a related patch set uploaded (by Hashar; author: Hashar):

[wmf-jvm-utils@master] Upgrade parent pom to latest version

https://gerrit.wikimedia.org/r/975353

Change 975352 merged by jenkins-bot:

[integration/config@master] Add Java 11 to wmf-jvm-utils

https://gerrit.wikimedia.org/r/975352

Change 975800 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Add Java 11 to wikimedia-event-utilities

https://gerrit.wikimedia.org/r/975800

Change 975800 merged by jenkins-bot:

[integration/config@master] Add Java 11 to wikimedia-event-utilities

https://gerrit.wikimedia.org/r/975800

hashar claimed this task.

I have added Java 11 to most repositories, filed some tasks to archive some repositories and have documented the few that are going to stick to Java 8. Not much more to do at this point, given we keep having Java 8 packages available :)

Change 974196 merged by jenkins-bot:

[integration/config@master] Add Java 11 jobs to analytics/gobblin-wmf

https://gerrit.wikimedia.org/r/974196

Change 975813 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Add back java8 to gate for analytics/gobblin-wmf

https://gerrit.wikimedia.org/r/975813

Change 975813 merged by jenkins-bot:

[integration/config@master] Add back java8 to gate for analytics/gobblin-wmf

https://gerrit.wikimedia.org/r/975813

Change 975353 merged by jenkins-bot:

[wmf-jvm-utils@master] Upgrade parent pom to latest version

https://gerrit.wikimedia.org/r/975353