Puppet failure on deployment-kafka-jumbo-3.deployment-prep.eqiad1.wikimedia.cloud
Closed, DuplicatePublic
Actions

Edit Task
Edit Related Tasks...
Create Subtask
Edit Parent Tasks
Edit Subtasks
Merge Duplicates In
Close As Duplicate
Edit Related Objects...
Edit Commits
Edit Mocks
Mute Notifications
Protect as security issue

Assigned To

None

Authored By

	dancy
	Aug 26 2021, 3:17 PM

Description

During puppet agent -t there are several instances of this error:

Cannot allocate memory - fork(2)

The machine has ~2GB RAM and no swap. Most of the memory is used by the two kafka processes (totaling ~1.5GB RSS). Looking at 'dmesg' I can see the oom-killer has been invoked several times.

How much RAM do the production kafka servers have?

Event Timeline

dancy created this task.Aug 26 2021, 3:17 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 26 2021, 3:17 PM

taavi closed this task as a duplicate of T286358: Puppet failing on deployment-kafka-jumbo-3 due to memory pressure.Aug 26 2021, 5:21 PM

Puppet failure on deployment-kafka-jumbo-3.deployment-prep.eqiad1.wikimedia.cloudClosed, DuplicatePublicActions

Description

Event Timeline

Puppet failure on deployment-kafka-jumbo-3.deployment-prep.eqiad1.wikimedia.cloud
Closed, DuplicatePublic
Actions