Page MenuHomePhabricator

Develop a RESTBase Cassandra cluster capacity planning document
Open, MediumPublic

Description

Needed, is a document which details acceptable levels of Cassandra cluster utilization, and prescribes timely, proactive measures as utilization increases. Ideally, this document would provide the means to plug-in current (or proposed) overall storage and utilization, and rate of growth, in order to project remaining runway to "actionables" (culling, upgrades, etc).

I propose we use: https://wikitech.wikimedia.org/wiki/Cassandra/CapacityPlanning

Event Timeline

Eevans created this task.Apr 15 2016, 7:16 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 15 2016, 7:16 PM
Eevans added a comment.EditedApr 29 2016, 9:49 PM

Worth noting:

At this time, the nodes in eqiad rack 'd' are in currently running cleanups, and their disk utilization is not indicative. However, utilization of the machines in racks 'a' and 'b' should be accurate, and average disk utilization there is already at 45%.

See also: T132615: Legacy data?

GWicke triaged this task as Medium priority.Oct 12 2016, 4:56 PM
GWicke added a project: Services (later).

See also T97692: RESTBase capacity planning for 2015/16 and T97710: Estimate storage capacity needed for storing all HTML revisions. Much of this is still relevant.

A major variable for capacity planning is the timing and actual efficiency of brotli compression (see T122028).

Eevans updated the task description. (Show Details)Jul 18 2018, 4:53 PM
Eevans moved this task from Backlog to In-Progress on the User-Eevans board.Jul 31 2018, 2:48 PM
Eevans moved this task from In-Progress to Next on the User-Eevans board.Jul 31 2018, 7:02 PM
Aklapper removed Eevans as the assignee of this task.Jun 19 2020, 4:23 PM

This task has been assigned to the same task owner for more than two years. Resetting task assignee due to inactivity, to decrease task cookie-licking and to get a slightly more realistic overview of plans. Please feel free to assign this task to yourself again if you still realistically work or plan to work on this task - it would be welcome!

For tips how to manage individual work in Phabricator (noisy notifications, lists of task, etc.), see https://phabricator.wikimedia.org/T228575#6237124 for available options.
(For the records, two emails were sent to assignee addresses before resetting assignees. See T228575 for more info and for potential feedback. Thanks!)