Page MenuHomePhabricator

Reduce memory allocation for kafkamon instances
Closed, ResolvedPublic3 Story Points

Description

These currently have 8GB assigned, but on average use significantly less, it should be fine to reduce them 4 GB at least.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 4 2019, 1:30 PM
MoritzMuehlenhoff triaged this task as Normal priority.Jun 4 2019, 1:30 PM
elukey added a subscriber: elukey.Jun 4 2019, 1:52 PM
elukey added a comment.EditedJul 1 2019, 7:29 AM

I would go down to 4G with (on ganeti1001):

sudo gnt-instance modify -B memory=4g kafkamon1001.eqiad.wmnet

Same thing for the codfw instance. From grafana it seems that we could go down even more, but since these hosts run Burrow (and we rely on it for Kafka lag metrics etc..) I'd be more cautious with the first step.

@akosiaris @MoritzMuehlenhoff does the above plan look ok?

elukey claimed this task.Jul 1 2019, 7:30 AM
elukey added a project: Analytics-Kanban.
elukey set the point value for this task to 3.
elukey added subscribers: herron, Ottomata.

I would go down to 4G with (on ganeti1001):

sudo gnt-instance modify -B memory=4g kafkamon1001.eqiad.wmnet

Same thing for the codfw instance. From grafana it seems that we could go down even more, but since these hosts run Burrow (and we rely on it for Kafka lag metrics etc..) I'd be more cautious with the first step.
@akosiaris @MoritzMuehlenhoff does the above plan look ok?

Already a net win, so fine by me. Thanks!

Mentioned in SAL (#wikimedia-operations) [2019-07-01T09:54:34Z] <elukey> reboot kafkamon2001 with 4g of dedicated ram (was 8g) - T224988

Mentioned in SAL (#wikimedia-operations) [2019-07-01T09:55:52Z] <elukey> reboot kafkamon1001 with 4g of dedicated ram (was 8g) - T224988

elukey moved this task from Next Up to Done on the Analytics-Kanban board.Jul 1 2019, 9:58 AM
Nuria closed this task as Resolved.Jul 3 2019, 10:23 PM