Page MenuHomePhabricator
Create Task
Maniphest T406172

Clean up ksqldb fallback from realtime
Open, Needs TriagePublic3 Estimated Story Points
Actions

Description

This aims to remove ksqldb-related codebase from realtime API. We will also remove ksqldb-related dependencies from other repos.

To do

  • Remove docker/ksqldb_services from general/cdtools. Realtime should build using usual docker file as other kafka services.
  • Remove ksqldb migration
  • Remove ksqldb from infrastructure/custom-images.
  • Remove all ksqldb specific grafana panel and metrics
  • Remove "docker-compose sunset" from realtime
  • Remove handler code ksqldbarticles. Remove fallback from /articles endpoint.
  • Remove libraries/ksqldb
  • Remove resolver/sql.go
  • Remove all ksqldb env variables and its usages
  • Clean up service-deploys as well.
  • Deploy to dev and prod

Acceptance criteria

  • All ksqld dependencies/handling removed from realtime
  • Integration test passes in dev and prod
  • check what needs to be done

Related Objects

Event Timeline

prabhat created this task.Oct 1 2025, 6:56 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 1 2025, 6:56 PM
prabhat set the point value for this task to 3.Oct 2 2025, 1:42 PM

Candidate for sprint 83 if next up is exhausted

JArguello-WMF renamed this task from Clean up ksqldb fallback from realtime to [pick if next up is clean] Clean up ksqldb fallback from realtime.Oct 2 2025, 1:44 PM
JArguello-WMF moved this task from To Be Estimated/To Be Discussed to Estimated /Discussed on the Wikimedia Enterprise board.
prabhat updated the task description. (Show Details)Oct 2 2025, 2:03 PM
prabhat updated the task description. (Show Details)Oct 3 2025, 5:59 PM
KMontalva-WMF updated the task description. (Show Details)Oct 28 2025, 12:02 PM
KMontalva-WMF subscribed.

First two action items done and tested in dev. Merge to prod next.

KMontalva-WMF updated the task description. (Show Details)Nov 3 2025, 2:37 PM

First two AIs are in prod, the newly checked ones are in dev.

JArguello-WMF updated the task description. (Show Details)Wed, Jan 28, 1:38 PM
JArguello-WMF edited projects, added Wikimedia-Enterprise-Kanban-On-Call; removed Wikimedia Enterprise.
JArguello-WMF moved this task from Next up to Blocked on the Wikimedia-Enterprise-Kanban-On-Call board.Fri, Jan 30, 8:10 PM
KMontalva-WMF renamed this task from [pick if next up is clean] Clean up ksqldb fallback from realtime to Clean up ksqldb fallback from realtime.Wed, Feb 4, 12:44 PM
KMontalva-WMF moved this task from Blocked to Next up on the Wikimedia-Enterprise-Kanban-On-Call board.
KMontalva-WMF claimed this task.Thu, Feb 5, 10:03 AM
KMontalva-WMF moved this task from Next up to Doing on the Wikimedia-Enterprise-Kanban-On-Call board.
KMontalva-WMF updated the task description. (Show Details)Mon, Feb 9, 11:58 AM
KMontalva-WMF moved this task from Doing to Done on the Wikimedia-Enterprise-Kanban-On-Call board.Mon, Feb 9, 12:30 PM
KMontalva-WMF updated the task description. (Show Details)

Filed https://phabricator.wikimedia.org/T416870 for one last part, which Renil suggested should be tracked separately. The service-deploys still have some references, but they are a bit tangled with that ticket, so I decided to leave them for now.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits