Page MenuHomePhabricator

Ensure Elastic stack works on bookworm
Open, MediumPublic

Description

Creating this ticket to:

  • Ensure all components of our Elastic stack work on Bookworm

Event Timeline

Gehel triaged this task as Medium priority.Dec 14 2023, 2:29 PM
Gehel moved this task from Incoming to OS Upgrade on the Data-Platform-SRE board.

Change 987859 had a related patch set uploaded (by Bking; author: Bking):

[operations/puppet@production] aptrepo: add Elastic-related components to bookworm repo

https://gerrit.wikimedia.org/r/987859

Change 987859 merged by Bking:

[operations/puppet@production] aptrepo: add Elastic-related components to bookworm repo

https://gerrit.wikimedia.org/r/987859

Change 988735 had a related patch set uploaded (by Bking; author: Bking):

[operations/puppet@production] elastic: assign prod role to elastic2088

https://gerrit.wikimedia.org/r/988735

Per today's pairing session with @RKemper , it looks like our current version of Elastic (7.10.2) does not support the version of Java provided by Debian Bookworm (17). As such, we'll need to make the Java 11 packages available on Bookworm.

We're doing something similar for Bullseye and Java 8 , so I think we'll just need to repeat the same steps for Bookworm/Java 11. @Muehlenhoff can you confirm/deny this?

If it's the same, it should be a very similar process as the apt repo patches/reprepro copying we've done earlier in this ticket.

Per today's pairing session with @RKemper , it looks like our current version of Elastic (7.10.2) does not support the version of Java provided by Debian Bookworm (17). As such, we'll need to make the Java 11 packages available on Bookworm.
We're doing something similar for Bullseye and Java 8 , so I think we'll just need to repeat the same steps for Bookworm/Java 11. @Muehlenhoff can you confirm/deny this?

That is correct. Bookworm only provides Java 17, so a backport of Java 11 for Bookworm would get provided by a separate repository component.

If it's the same, it should be a very similar process as the apt repo patches/reprepro copying we've done earlier in this ticket.

Not really, no :-)

We'll need to "back"port/rebuild OpenJDK 11 with the C/C++ toolchain from Bookworm (along with some infrastructure packages like jtreg and jtest used during build). That's not a quick undertaking but I think I can make it work that these packages are available in a separate repo component in 2-3 weeks. And obviously once the initial build is completed we'll need to rebase the quartely Java security releases on this build (sincce we can't just use what Debian releases).

That's a bit of effort, but it's perfectly fine; it'll unblock moving an important piece of infrastructure towards Bookworm (and we'll most probably need this for other services as well, at this point Hadoop is still working on support for Java 11 (https://issues.apache.org/jira/browse/HADOOP-16795)...)

Thanks for the update! I'll set this to blocked. No rush or anything, but do reach out when the packages are ready or if we can do anything to help.

We need to evaluate the amount of work needed to migrate to OpenSearch instead of backporting OpenJDK. We will need to do that migration eventually anyway.

@MoritzMuehlenhoff : we're not working on the Bookworm upgrade yet. We're probably going to need Java 11 on Bookworm when we do, but we'll confirm once we get started.

Gehel removed bking as the assignee of this task.Feb 9 2024, 10:48 AM

Change #988735 abandoned by Bking:

[operations/puppet@production] elastic: assign prod role to elastic2088

Reason:

no longer trying bookworm

https://gerrit.wikimedia.org/r/988735