Page MenuHomePhabricator
Create Task
Maniphest T436023

Tune WDQS backend storage params, and document the decision
Closed, ResolvedPublic1 Estimated Story Points
Actions

Description

Now that we can updating the index more regularly and that the qlever index rebuilds are released, we have a better idea of what the storage needs are for the system.

We ran into an issue where there wasn't room to download a third index given the delta-triples plus two existing indices in a 600GiB volume. There might be additional issues like this, but nothing obvious comes to mind now so we will file follow up tickets if they arise.

Acceptance criterion:

  • document why 900 GiB in the doc mentioned here: T438165

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
Increase WDQS backend pod storage to 900GiB.Lericksonoperations/deployment-chartsmaster+3 -2
Customize query in gerrit

Related Objects
Search...

Event Timeline

lerickson created this task.Aug 25 2026, 9:47 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 25 2026, 9:47 PM
gerritbot added a comment.Aug 25 2026, 9:48 PM

Change #1329388 had a related patch set uploaded (by Lerickson; author: Lerickson):

[operations/deployment-charts@master] Increase WDQS backend pod storage to 900GiB.

https://gerrit.wikimedia.org/r/1329388

gerritbot added a project: Patch-For-Review.Aug 25 2026, 9:48 PM
gerritbot added a comment.Aug 26 2026, 1:53 PM

Change #1329388 merged by jenkins-bot:

[operations/deployment-charts@master] Increase WDQS backend pod storage to 900GiB.

https://gerrit.wikimedia.org/r/1329388

Maintenance_bot removed a project: Patch-For-Review.Aug 26 2026, 2:31 PM
trueg renamed this task from [NEEDS GROOMING] Tune WDQS backend storage and memory params to Tune WDQS backend storage and memory params.Sep 2 2026, 2:04 PM
trueg added a subtask: T436301: wdqs backend pods: upsize topoLVM from 600GiB->900GiB.
trueg claimed this task.Sep 2 2026, 2:07 PM
trueg renamed this task from Tune WDQS backend storage and memory params to Tune WDQS backend storage params.Sep 2 2026, 2:10 PM
trueg updated the task description. (Show Details)
trueg created subtask T436802: Disable backups on wdqs-backend-init.Sep 2 2026, 2:13 PM
trueg changed the status of subtask T436802: Disable backups on wdqs-backend-init from Open to In Progress.Sep 4 2026, 9:26 AM
trueg changed the status of subtask T436301: wdqs backend pods: upsize topoLVM from 600GiB->900GiB from Open to In Progress.
trueg moved this task from Backlog to In Engineering on the Wikidata Platform Team (Sprint 08 (2026/08/04)) board.Sep 4 2026, 11:27 AM
trueg moved this task from In Engineering to In Code Review on the Wikidata Platform Team (Sprint 08 (2026/08/04)) board.Sep 7 2026, 10:49 AM
BTracy-WMF closed subtask T436802: Disable backups on wdqs-backend-init as Resolved.Sep 9 2026, 8:42 PM
BTracy-WMF closed subtask T436301: wdqs backend pods: upsize topoLVM from 600GiB->900GiB as Resolved.Sep 9 2026, 8:44 PM
trueg renamed this task from Tune WDQS backend storage params to [NEEDS GROOMING] Tune WDQS backend storage params.Sep 11 2026, 11:04 AM
trueg removed trueg as the assignee of this task.
trueg moved this task from Sprint 08 (2026/08/04) to Sprint 09 (2026/09/08) on the Wikidata Platform Team board.
trueg edited projects, added: Wikidata Platform Team (Sprint 09 (2026/09/08)); removed: Wikidata Platform Team (Sprint 08 (2026/08/04)).
trueg moved this task from Backlog to In Engineering on the Wikidata Platform Team (Sprint 09 (2026/09/08)) board.
trueg moved this task from In Engineering to Backlog on the Wikidata Platform Team (Sprint 09 (2026/09/08)) board.
trueg subscribed.
lerickson renamed this task from [NEEDS GROOMING] Tune WDQS backend storage params to Tune WDQS backend storage params, and document the decision.Wed, Sep 16, 2:00 PM
lerickson updated the task description. (Show Details)
lerickson set the point value for this task to 1.
gmodena subscribed.EditedWed, Sep 23, 1:25 PM

document why 900 GiB in the doc mentioned here: T438165

@lerickson IIRC the AC was to document the adoption of TopoLVM in a decision log (done here ). For the sizing itself, I think we properly documented in the right place (where the change is applied).

Currently we have 2TBs per k8s node. Each node has two PODs deployed (main and scholarly), so 900GB gives enough headroom for updating the indexes and persisting delta triples) of both PODs.

gmodena claimed this task.Wed, Sep 23, 1:25 PM
gmodena moved this task from Backlog to In Engineering on the Wikidata Platform Team (Sprint 09 (2026/09/08)) board.
lerickson added a comment.Wed, Sep 23, 1:55 PM

LGTM @gmodena , thanks. Agreed that 900 GiB can be documented at the source. However, I think it still says this is based on 3 indices (1 live, 1 backup, 1 being downloaded), whereas we'd decided to have no backup. Am I misremembering?
https://gerrit.wikimedia.org/r/plugins/gitiles/operations/deployment-charts/+/refs/heads/master/helmfile.d/dse-k8s-services/_wdqs_common_/values.yaml#7

gmodena moved this task from In Engineering to In Code Review on the Wikidata Platform Team (Sprint 09 (2026/09/08)) board.Thu, Sep 24, 8:37 AM
gmodena moved this task from In Code Review to Needs Sign-off on the Wikidata Platform Team (Sprint 09 (2026/09/08)) board.Wed, Sep 30, 3:15 PM
BTracy-WMF closed this task as Resolved.Wed, Oct 7, 7:38 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits