Maniphest T401725

Deploy mediawiki-event-enrichment Flink jobs running 1.20
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Ottomata
	Aug 12 2025, 5:22 PM

Tags

Referenced Files

None

Subscribers

Description

Code prep work for Flink 1.20 has already been done in T377130: Bump eventutilities to support flink 1.20 and T374359: Update eventutilities_python wrappers to support Flink 1.20
Flink 1.20 is available in our production images as part of T398159: SUP: Use flink 1.20.1
The flink-k8s-operator has been upgraded in T398162: Flink: Update k8s operator to 1.12.0
T400296: Flink images should be build on top of openjdk will get us openjdk and (debian bookworm) for debugging

Once all of the above are done, we can proceed with releasing deploying mediawiki-event-enrichment based jobs.

Done is

mediawiki-event-enrichment released using Flink 1.20.x
mw-page-content-change-enrich is deployed in wikikube
mw-content-history-reconcile-enrich* is deployed in dse-k8s

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	[mw-enrichment] Bump page change schema to 1.3.0 to pick up user_central_id	operations/deployment-charts	master	+6 -6
	[mw-enrichment] Bump to v1.42.0 and Flink 1.20	operations/deployment-charts	master	+6 -6

Customize query in gerrit

Related Changes in GitLab:

	Title	Reference	Author	Source Branch	Dest Branch
	Bump to Flink 1.20	repos/data-engineering/mediawiki-event-enrichment!97	tchin	bump-flink-1.20	main

Customize query in GitLab

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T376812 EPIC: Update flink jobs to support Flink 1.20
		Resolved		gmodena	T374359 Update eventutilities_python wrappers to support Flink 1.20
		Open		None	T401692 EPIC: Migrate Data Platform SRE-owned hosts to Bookworm or later
		Open		MoritzMuehlenhoff	T401694 Make Java 11 available in our Debian Bookworm repos
		Declined		None	T400296 Flink images should be build on top of openjdk
		Resolved		tchin	T401725 Deploy mediawiki-event-enrichment Flink jobs running 1.20
		Resolved		tchin	T397330 mediawiki.content_history: flink applications experiencing frequent restarts due to JobManager OOMs
		Resolved		BTullis	T402886 Alert in need of triage: KubernetesContainerReachingMemoryLimit (instance wikikube-worker1119.eqiad.wmnet)

Event Timeline

Ottomata created this task.Aug 12 2025, 5:22 PM

Ottomata added a subtask: T397330: mediawiki.content_history: flink applications experiencing frequent restarts due to JobManager OOMs.

Ottomata added a parent task: T400296: Flink images should be build on top of openjdk.Aug 12 2025, 5:24 PM

Ottomata mentioned this in T376812: EPIC: Update flink jobs to support Flink 1.20.

Ottomata mentioned this in T397336: Investigate mw-page-content-change Flink memory alerts.Aug 12 2025, 5:32 PM

Ottomata merged a task: T400623: update mediawiki-event-enrichment base docker image.Aug 12 2025, 5:59 PM

Ottomata moved this task from Incoming (new tickets) to Q1 FY25/26 July 1st - September 30th on the Data-Engineering board.

Ottomata edited projects, added Data-Engineering (Q1 FY25/26 July 1st - September 30th); removed Data-Engineering.

Ottomata moved this task from Next Up to Blocked/Paused on the Data-Engineering (Q1 FY25/26 July 1st - September 30th) board.

Ottomata reassigned this task from Ottomata to tchin.Sep 22 2025, 3:10 PM

Ahoelzl moved this task from Blocked/Paused to In progress on the Data-Engineering (Q1 FY25/26 July 1st - September 30th) board.Sep 22 2025, 3:39 PM

BTracy-WMF subscribed.Sep 29 2025, 2:48 PM

Ottomata added a project: Event-Platform.Sep 29 2025, 4:52 PM

tchin opened https://gitlab.wikimedia.org/repos/data-engineering/mediawiki-event-enrichment/-/merge_requests/97

Draft: Bump to Flink 1.20

Ahoelzl edited projects, added Data-Engineering (Q2 FY25/26 October 1st - December 31th); removed Data-Engineering (Q1 FY25/26 July 1st - September 30th).Oct 9 2025, 12:20 AM

Ahoelzl moved this task from Next Up to In progress on the Data-Engineering (Q2 FY25/26 October 1st - December 31th) board.

tchin merged https://gitlab.wikimedia.org/repos/data-engineering/mediawiki-event-enrichment/-/merge_requests/97

Bump to Flink 1.20

Maintenance_bot removed a project: Patch-For-Review.Oct 9 2025, 4:31 PM

Change #1195223 had a related patch set uploaded (by TChin; author: TChin):

[operations/deployment-charts@master] [mw-enrichment] Bump to v1.42.0 and Flink 1.20

https://gerrit.wikimedia.org/r/1195223

gerritbot added a project: Patch-For-Review.Oct 10 2025, 1:55 PM

Change #1195223 merged by jenkins-bot:

[operations/deployment-charts@master] [mw-enrichment] Bump to v1.42.0 and Flink 1.20

https://gerrit.wikimedia.org/r/1195223

Maintenance_bot removed a project: Patch-For-Review.Oct 14 2025, 6:31 PM

Change #1196179 had a related patch set uploaded (by TChin; author: TChin):

[operations/deployment-charts@master] [mw-enrichment] Bump page change schema to 1.3.0 to pick up user_central_id

https://gerrit.wikimedia.org/r/1196179

gerritbot added a project: Patch-For-Review.Oct 14 2025, 9:21 PM

Change #1196179 merged by jenkins-bot:

[operations/deployment-charts@master] [mw-enrichment] Bump page change schema to 1.3.0 to pick up user_central_id

https://gerrit.wikimedia.org/r/1196179

Maintenance_bot removed a project: Patch-For-Review.Oct 15 2025, 1:31 PM

mforns moved this task from In progress to Done on the Data-Engineering (Q2 FY25/26 October 1st - December 31th) board.Oct 21 2025, 3:08 PM

Ahoelzl closed this task as Resolved.Oct 24 2025, 9:53 PM

Ahoelzl closed subtask T397330: mediawiki.content_history: flink applications experiencing frequent restarts due to JobManager OOMs as Resolved.