Cassandra on deployment-sessionstore04 is being frequently killed by the kernel (OOM)
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Eevans
	Jan 20 2023, 4:15 PM

Description

The Cassandra process on deployment-sessionstore04 is being periodically killed by the kernel.

[2664642.115186] Out of memory: Kill process 12005 (java) score 697 or sacrifice child
[2664642.116467] Killed process 12005 (java) total-vm:2940372kB, anon-rss:1398272kB, file-rss:25096kB, shmem-rss:0kB
[2664642.127207] oom_reaper: reaped process 12005 (java), now anon-rss:1247036kB, file-rss:26880kB, shmem-rss:0kB

It is eventually restarted by puppet-agent, but not before creating the potential for test failures (see T324128).

Related Objects

Mentioned In: T324128: Can not log in, log out, or save edits to the beta cluster (session failures)
T327524: Kask: gocql pool errors after repeated Cassandra outages
Mentioned Here: T324128: Can not log in, log out, or save edits to the beta cluster (session failures)

Event Timeline

Eevans created this task.Jan 20 2023, 4:15 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 20 2023, 4:15 PM

Eevans triaged this task as Medium priority.Jan 20 2023, 4:15 PM

Eevans renamed this task from Cassandra on deployment-sessionstore04 is being frequently killed by the kernel (oom) to Cassandra on deployment-sessionstore04 is being frequently killed by the kernel (OOM).Jan 20 2023, 4:19 PM

Eevans updated the task description. (Show Details)

Eevans mentioned this in T327524: Kask: gocql pool errors after repeated Cassandra outages.Jan 20 2023, 4:25 PM

Eevans mentioned this in T324128: Can not log in, log out, or save edits to the beta cluster (session failures).Jan 20 2023, 4:34 PM

TheresNoTime subscribed.Feb 13 2023, 10:49 AM

This doesn't seem to be happening anymore; Closing

Cassandra on deployment-sessionstore04 is being frequently killed by the kernel (OOM)Closed, ResolvedPublicActions

Description

Related Objects

Event Timeline

Cassandra on deployment-sessionstore04 is being frequently killed by the kernel (OOM)
Closed, ResolvedPublic
Actions