Page MenuHomePhabricator

wdqs-categories migration: decide where to migrate
Closed, ResolvedPublic

Description

wdqs-categories , a separate blazegraph instance which powers the deepcat service , is currently deployed alongside the primary blazegraph instance on all legacy WDQS (aka "full graph") hosts.

Since we split the primary blazegraph instance itself in T337013, we've started wondering about how to handle wdqs-categories. The legacy WDQS hosts will eventually be retired (no sooner than 6 months from now), so we need to find a new home for wdqs-categories.

Creating this ticket to:

  • decide whether or not to migrate wdqs-categories to VMs, baremetal hosts, Kubernetes, WMCS or any other available options

Once we decide, we'll open a new ticket for the implementation.

Event Timeline

bking renamed this task from wdqs-categories migration: decide between Kubernetes and VMs to wdqs-categories migration: decide between VMs, baremetal hosts, or Kubernetes.Sep 23 2024, 2:36 PM
bking updated the task description. (Show Details)
bking renamed this task from wdqs-categories migration: decide between VMs, baremetal hosts, or Kubernetes to wdqs-categories migration: decide where to migrate.Sep 23 2024, 3:58 PM
bking updated the task description. (Show Details)
Gehel triaged this task as Medium priority.Sep 24 2024, 2:27 PM
Gehel moved this task from Incoming to Scratch on the Data-Platform-SRE board.
Gehel moved this task from Scratch to Toil / Automation on the Data-Platform-SRE board.

Note that if we do want to run in Kubernetes, we need to build a wdqs image, similar to what WMDE does here

bking claimed this task.

Per last week's offsite, we have decided the following:

  • Short-term, categories will live on main and internal-main wdqs hosts (ref T385896)
  • Long-term, categories will migrate to Kubernetes. Whether that's on the wikikube cluster or the dse-k8s-cluster is yet to be determined. I'll record that decision in a future ticket once the decision has been made.

For now though, we have made our decision and thus I'm closing out this ticket.