Page MenuHomePhabricator

Mutation of <x> bytes is too large..., exceptions
Closed, DeclinedPublic

Description

We are seeing a good deal of exceptions for mutations that exceed the maximum size (on the order of hundreds per hour).

https://logstash.wikimedia.org/goto/e33d225c98827ecc58203931a4899c21

Each one that I've examined thus far was generated by a read-repair, like the example below:

java.lang.IllegalArgumentException: Mutation of 62111594 bytes is too large for the maximum size of 16777216
	at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:264) ~[apache-cassandra-2.2.6.jar:2.2.6]
	at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:389) ~[apache-cassandra-2.2.6.jar:2.2.6]
	at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:366) ~[apache-cassandra-2.2.6.jar:2.2.6]
	at org.apache.cassandra.db.Mutation.apply(Mutation.java:214) ~[apache-cassandra-2.2.6.jar:2.2.6]
	at org.apache.cassandra.db.ReadRepairVerbHandler.doVerb(ReadRepairVerbHandler.java:28) ~[apache-cassandra-2.2.6.jar:2.2.6]
	at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67) ~[apache-cassandra-2.2.6.jar:2.2.6]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_102]
	at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) ~[apache-cassandra-2.2.6.jar:2.2.6]
	at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136) [apache-cassandra-2.2.6.jar:2.2.6]
	at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) [apache-cassandra-2.2.6.jar:2.2.6]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]

Event Timeline

GWicke added a project: Services (next).

This issue is quite old, and it does not appear to be an issue for the new storage strategy; Let's close