Change #1173349 had a related patch set uploaded (by Gmodena; author: Gmodena):

[operations/docker-images/production-images@master] flink: update image to bookworm

https://gerrit.wikimedia.org/r/1173349

According to the documentation, Flink 1.20 can run on JDK 11 (which we don't have for Bookworm, but we could build), or experimentally on JDK 17, which we already do have for bookworm.

@gmodena the doc states:

These Flink features have not been tested with Java 17:

• Hive connector
• Hbase 1.x connector

Are we using any of these?

Message from @MoritzMuehlenhoff

so I'd say let's simply give 17 a shot and if there's issues we can still consider building Java 11 for Bookworm (it's doable, but incurs some long term cost since we need to respin it every quarter for the Java security cycles)

We might need Java 11 on bookworm in order to support Hadoop, anyway.
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=100827883#HadoopJavaVersions-SupportedJavaVersions

Apache Hadoop 3.3 and upper supports Java 8 and Java 11 (runtime only)

• Please compile Hadoop with Java 8. Compiling Hadoop with Java 11 is not supported

There is no way that Hadoop will support Java 17 and I am guessing that we won't want to continue using Java 8 on bookworm and trixie, if we can avoid it.

What is the urgency for bookworm? I think there is a little bit of urgency for Flink 1.20 and for T400296 but really for better memory and other debugging capabilities.

Could we just do Flink 1.20 on bullseye with Java 11?

In T400600#11077978, @Ottomata wrote:

What is the urgency for bookworm?

https://wikitech.wikimedia.org/wiki/Operating_system_upgrade_policy#Current_effective_sunset_dates_for_each_distro_(rounded_to_quarters)

Oh, wow, sooner than I thought. Okay, let's get Java 11 .devs then. Thanks.

I've created T401694 to get Java 11 into the Bookworm repos. Once that's complete, we should be able to build the Flink image on bookworm.

@JAllemandou @dcausse and myself looked at this ticket again. In the wake of pausing the Hadoop upgrade (or at least considering it), there's a world in which we don't need to build JDK11 for bookworm. @dcausse and I are willing to test our flink apps on JDK 17 and report back.

Change #1187416 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/docker-images/production-images@master] flink: build flink 1.20 on top of bookworm/jdk 17

https://gerrit.wikimedia.org/r/1187416

Change #1187416 merged by Brouberol:

[operations/docker-images/production-images@master] flink: build flink 1.20 on top of bookworm/jdk 17

https://gerrit.wikimedia.org/r/1187416

* docker-registry.discovery.wmnet/flink:1.20.1-wmf1-20250911
== Step 1: building images ==
* Built image docker-registry.discovery.wmnet/flink:1.20.1-wmf1-20250911
== Step 2: publishing ==
Successfully published image docker-registry.discovery.wmnet/flink:1.20.1-wmf1-20250911
== Build done! ==
You can see the logs at ./docker-pkg-build.log

We now have docker-registry.discovery.wmnet/flink:1.20.1-wmf1-20250911 published, which is based on bookworm, and contains the JRE 17.

~ ❯ docker run -it docker-registry.wikimedia.org/flink:1.20.1-wmf1-20250911 bash
flink@eab6327f2e07:/usr/local/lib/python3.11/dist-packages/pyflink$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
flink@eab6327f2e07:/usr/local/lib/python3.11/dist-packages/pyflink$ java -version
openjdk version "17.0.16" 2025-07-15
OpenJDK Runtime Environment (build 17.0.16+8-Debian-1deb12u1)
OpenJDK 64-Bit Server VM (build 17.0.16+8-Debian-1deb12u1, mixed mode, sharing)

I'll let this ticket open and mark it as waiting while @dcausse performs some tests with this image.

@brouberol thanks! I have T404417 for testing, quick note regarding the image tag, IIUC in https://gerrit.wikimedia.org/r/c/operations/docker-images/production-images/+/1187416/2/images/flink/flink/Dockerfile.template you upgraded to flink 1.20.2 but the image tag still refers to 1.20.1, checking the image it's actually 1.20.2. No big deal for now since we're just testing.

Oops, you're right. I'll rebuild it with a new tag.

Change #1187673 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/docker-images/production-images@master] Update tag for flink 1.20.2 image

https://gerrit.wikimedia.org/r/1187673

Change #1187673 merged by Brouberol:

[operations/docker-images/production-images@master] Update tag for flink 1.20.2 image

https://gerrit.wikimedia.org/r/1187673

@brouberol thanks!

This image seems to work well:

• a test search job is running with it since a couple weeks
• Peter just merged https://gitlab.wikimedia.org/repos/search-platform/cirrus-streaming-updater/-/merge_requests/196/diffs#3dedbbcda032aef89251d2546ba01531825cb2fc which we'll deploy soon

Thanks! Closing this one.

Change #1173349 abandoned by Gmodena:

[operations/docker-images/production-images@master] flink: update image to bookworm

https://gerrit.wikimedia.org/r/1173349