Needed, is a document which details acceptable levels of Cassandra cluster utilization, and prescribes timely, proactive measures as utilization increases. Ideally, this document would provide the means to plug-in current (or proposed) overall storage and utilization, and rate of growth, in order to project remaining runway to "actionables" (culling, upgrades, etc).
I propose we use: https://wikitech.wikimedia.org/wiki/Cassandra/CapacityPlanning
Worth noting:
At this time, the nodes in eqiad rack 'd' are in currently running cleanups, and their disk utilization is not indicative. However, utilization of the machines in racks 'a' and 'b' should be accurate, and average disk utilization there is already at 45%.
See also: T132615: Legacy data?
See also T97692: RESTBase capacity planning for 2015/16 and T97710: Estimate storage capacity needed for storing all HTML revisions. Much of this is still relevant.
A major variable for capacity planning is the timing and actual efficiency of brotli compression (see T122028).
This task has been assigned to the same task owner for more than two years. Resetting task assignee due to inactivity, to decrease task cookie-licking and to get a slightly more realistic overview of plans. Please feel free to assign this task to yourself again if you still realistically work or plan to work on this task - it would be welcome!
For tips how to manage individual work in Phabricator (noisy notifications, lists of task, etc.), see https://phabricator.wikimedia.org/T228575#6237124 for available options.
(For the records, two emails were sent to assignee addresses before resetting assignees. See T228575 for more info and for potential feedback. Thanks!)