New hardware for production core mysql cluster
Closed, DuplicatePublic
Actions

Assigned To

None

Authored By

	jcrespo
	Jul 24 2015, 4:53 PM

Description

s3 is about to hit the 90% limit, although this time is giving us a 6 month-1 year advance notice:

s4 also is complaining about lack of space.

Mentioned In: T110504: defragment db1015, db1035 and db1027
T95501: Fix causes of replica lag and get it to under 5 seconds at peak
T87277: db1021 %iowait up
T105843: new external storage cluster(s)
Mentioned Here: T110504: defragment db1015, db1035 and db1027
T104278: E7. Run populateContentModel.php for page on all WMF wikis
P1057 Disk space on s3

jcrespo raised the priority of this task from to Low.

jcrespo updated the task description. (Show Details)

The latest bump is probably caused by T104278.

db1035 died.

@Springle We need to spec a hardware replacement to be used for production hosts and send it to @RobH

jcrespo renamed this task from New s3 production cluster for mysql to New hardware for production core mysql cluster.Aug 7 2015, 9:46 AM

jcrespo updated the task description. (Show Details)

jcrespo set Security to None.

In T106847#1505661, @jcrespo wrote:

We need to spec a hardware replacement to be used for production hosts and send it to @RobH

Has that happened in the meantime?

Aklapper, 4 months to full failure? that is too much time for operator standards! :-)

I designed with Sean's help an initial spec, have to ask for initial quotes still.

s3 is in critical state (see T110504), send procurement soon.

jcrespo closed this task as a duplicate of Unknown Object (Task).Nov 16 2015, 11:00 AM