Page MenuHomePhabricator
Paste P8960

/var/log/cassandra/debug-b.log
ActivePublic

Authored by Eevans on Aug 22 2019, 4:33 PM.
Tags
None
Referenced Files
F30064746: raw.txt
Aug 22 2019, 4:33 PM
Subscribers
None
DEBUG [COMMIT-LOG-ALLOCATOR] 2019-08-22 15:00:05,844 AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a fresh one
DEBUG [NativePoolCleaner] 2019-08-22 15:00:19,233 ColumnFamilyStore.java:1308 - Flushing largest CFS(Keyspace='wikipedia_T_parsoid', ColumnFamily='data') to free up room. Used total: 0.01/0.50, live: 0.01/0.50, flushing: 0.00/0.00, this: 0.00/0.09
DEBUG [NativePoolCleaner] 2019-08-22 15:00:19,234 ColumnFamilyStore.java:918 - Enqueuing flush of data: 164.063KiB (0%) on-heap, 288.933MiB (9%) off-heap
DEBUG [PerDiskMemtableFlushWriter_0:9424] 2019-08-22 15:00:19,287 Memtable.java:456 - Writing Memtable-data@634761634(86.039MiB serialized bytes, 1065 ops, 0%/9% of on/off-heap limit), flushed range = (min(-9223372036854775808), max(-1462500924879604616)]
DEBUG [PerDiskMemtableFlushWriter_1:9423] 2019-08-22 15:00:19,287 Memtable.java:456 - Writing Memtable-data@634761634(86.039MiB serialized bytes, 1065 ops, 0%/9% of on/off-heap limit), flushed range = (max(-1462500924879604616), max(3778679129973661054)]
DEBUG [PerDiskMemtableFlushWriter_2:9424] 2019-08-22 15:00:19,287 Memtable.java:456 - Writing Memtable-data@634761634(86.039MiB serialized bytes, 1065 ops, 0%/9% of on/off-heap limit), flushed range = (max(3778679129973661054), max(9223372036854775807)]
DEBUG [PerDiskMemtableFlushWriter_1:9423] 2019-08-22 15:00:19,395 Memtable.java:485 - Completed flushing /srv/sdb4/cassandra-b/data/wikipedia_T_parsoid/data-db05bba065db11e9bc6c8958654eb0a6/mc-49095-big-Data.db (23.455MiB) for commitlog position CommitLogPosition(segmentId=1565021954783, position=24182044)
DEBUG [PerDiskMemtableFlushWriter_2:9424] 2019-08-22 15:00:19,423 Memtable.java:485 - Completed flushing /srv/sdc4/cassandra-b/data/wikipedia_T_parsoid/data-db05bba065db11e9bc6c8958654eb0a6/mc-49096-big-Data.db (30.703MiB) for commitlog position CommitLogPosition(segmentId=1565021954783, position=24182044)
DEBUG [PerDiskMemtableFlushWriter_0:9424] 2019-08-22 15:00:19,437 Memtable.java:485 - Completed flushing /srv/sda4/cassandra-b/data/wikipedia_T_parsoid/data-db05bba065db11e9bc6c8958654eb0a6/mc-49094-big-Data.db (31.892MiB) for commitlog position CommitLogPosition(segmentId=1565021954783, position=24182044)
DEBUG [MemtableFlushWriter:9423] 2019-08-22 15:00:19,516 ColumnFamilyStore.java:1216 - Flushed to [BigTableReader(path='/srv/sda4/cassandra-b/data/wikipedia_T_parsoid/data-db05bba065db11e9bc6c8958654eb0a6/mc-49094-big-Data.db'), BigTableReader(path='/srv/sdb4/cassandra-b/data/wikipedia_T_parsoid/data-db05bba065db11e9bc6c8958654eb0a6/mc-49095-big-Data.db'), BigTableReader(path='/srv/sdc4/cassandra-b/data/wikipedia_T_parsoid/data-db05bba065db11e9bc6c8958654eb0a6/mc-49096-big-Data.db')] (3 sstables, 22.548MiB), biggest 8.510MiB, smallest 6.160MiB
DEBUG [CompactionExecutor:45589] 2019-08-22 15:00:19,517 CompactionTask.java:155 - Compacting (8ee4bfc0-c4ed-11e9-ad4b-e11861b6015e) [/srv/sda4/cassandra-b/data/wikipedia_T_parsoid/data-db05bba065db11e9bc6c8958654eb0a6/mc-49094-big-Data.db:level=0, /srv/sda4/cassandra-b/data/wikipedia_T_parsoid/data-db05bba065db11e9bc6c8958654eb0a6/mc-49091-big-Data.db:level=0, /srv/sda4/cassandra-b/data/wikipedia_T_parsoid/data-db05bba065db11e9bc6c8958654eb0a6/mc-49086-big-Data.db:level=0, /srv/sda4/cassandra-b/data/wikipedia_T_parsoid/data-db05bba065db11e9bc6c8958654eb0a6/mc-49085-big-Data.db:level=0, ]
DEBUG [COMMIT-LOG-ALLOCATOR] 2019-08-22 15:00:19,759 AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a fresh one
DEBUG [COMMIT-LOG-ALLOCATOR] 2019-08-22 15:00:20,776 AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a fresh one
WARN [PERIODIC-COMMIT-LOG-SYNCER] 2019-08-22 15:00:20,916 NoSpamLogger.java:94 - Out of 0 commit log syncs over the past 0.00s with average duration of Infinityms, 1 have exceeded the configured commit interval by an average of 39.99ms
DEBUG [NativePoolCleaner] 2019-08-22 15:00:20,918 ColumnFamilyStore.java:1308 - Flushing largest CFS(Keyspace='enwiki_T_parsoid', ColumnFamily='data') to free up room. Used total: 0.01/0.50, live: 0.01/0.50, flushing: 0.00/0.00, this: 0.00/0.18
DEBUG [NativePoolCleaner] 2019-08-22 15:00:20,918 ColumnFamilyStore.java:918 - Enqueuing flush of data: 14.648KiB (0%) on-heap, 561.592MiB (18%) off-heap
DEBUG [PerDiskMemtableFlushWriter_2:9424] 2019-08-22 15:00:21,020 Memtable.java:456 - Writing Memtable-data@411722(14.848MiB serialized bytes, 244 ops, 0%/22% of on/off-heap limit), flushed range = (max(3778679129973661054), max(9223372036854775807)]
DEBUG [PerDiskMemtableFlushWriter_1:9423] 2019-08-22 15:00:21,020 Memtable.java:456 - Writing Memtable-data@411722(14.848MiB serialized bytes, 244 ops, 0%/22% of on/off-heap limit), flushed range = (max(-1462500924879604616), max(3778679129973661054)]
DEBUG [PerDiskMemtableFlushWriter_0:9424] 2019-08-22 15:00:21,020 Memtable.java:456 - Writing Memtable-data@411722(14.848MiB serialized bytes, 244 ops, 0%/22% of on/off-heap limit), flushed range = (min(-9223372036854775808), max(-1462500924879604616)]
DEBUG [PerDiskMemtableFlushWriter_1:9423] 2019-08-22 15:00:21,028 Memtable.java:485 - Completed flushing /srv/sdb4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40013-big-Data.db (1.217MiB) for commitlog position CommitLogPosition(segmentId=1565021954785, position=13511880)
DEBUG [PerDiskMemtableFlushWriter_2:9424] 2019-08-22 15:00:21,051 Memtable.java:485 - Completed flushing /srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40014-big-Data.db (6.381MiB) for commitlog position CommitLogPosition(segmentId=1565021954785, position=13511880)
DEBUG [PerDiskMemtableFlushWriter_0:9424] 2019-08-22 15:00:21,065 Memtable.java:485 - Completed flushing /srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40012-big-Data.db (7.251MiB) for commitlog position CommitLogPosition(segmentId=1565021954785, position=13511880)
DEBUG [MemtableFlushWriter:9423] 2019-08-22 15:00:21,083 ColumnFamilyStore.java:1216 - Flushed to [BigTableReader(path='/srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40012-big-Data.db'), BigTableReader(path='/srv/sdb4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40013-big-Data.db'), BigTableReader(path='/srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40014-big-Data.db')] (3 sstables, 3.175MiB), biggest 1.568MiB, smallest 367.798KiB
DEBUG [CompactionExecutor:45588] 2019-08-22 15:00:21,084 CompactionTask.java:155 - Compacting (8fd401c0-c4ed-11e9-ad4b-e11861b6015e) [/srv/sdb4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40013-big-Data.db:level=0, /srv/sdb4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40009-big-Data.db:level=0, /srv/sdb4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40003-big-Data.db:level=0, /srv/sdb4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40001-big-Data.db:level=0, ]
WARN [Service Thread] 2019-08-22 15:00:22,757 GCInspector.java:282 - G1 Young Generation GC in 1121ms. G1 Eden Space: 2566914048 -> 0; G1 Old Gen: 7893013408 -> 4274730688; G1 Survivor Space: 150994944 -> 0;
DEBUG [COMMIT-LOG-ALLOCATOR] 2019-08-22 15:00:22,796 AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a fresh one
DEBUG [ReadRepairStage:609645] 2019-08-22 15:00:22,797 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609653] 2019-08-22 15:00:22,887 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609656] 2019-08-22 15:00:23,014 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609660] 2019-08-22 15:00:23,045 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609655] 2019-08-22 15:00:23,059 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609631] 2019-08-22 15:00:23,079 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609639] 2019-08-22 15:00:23,188 ReadCallback.java:242 - Digest mismatch:
DEBUG [COMMIT-LOG-ALLOCATOR] 2019-08-22 15:00:23,190 AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a fresh one
DEBUG [ReadRepairStage:609649] 2019-08-22 15:00:23,260 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609645] 2019-08-22 15:00:23,281 ReadCallback.java:242 - Digest mismatch:
DEBUG [NativePoolCleaner] 2019-08-22 15:00:23,297 ColumnFamilyStore.java:1308 - Flushing largest CFS(Keyspace='enwiki_T_parsoid', ColumnFamily='data') to free up room. Used total: 0.01/0.50, live: 0.01/0.50, flushing: 0.00/0.00, this: 0.00/0.18
DEBUG [NativePoolCleaner] 2019-08-22 15:00:23,298 ColumnFamilyStore.java:918 - Enqueuing flush of data: 6.152KiB (0%) on-heap, 566.784MiB (18%) off-heap
DEBUG [ReadRepairStage:609654] 2019-08-22 15:00:23,338 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609641] 2019-08-22 15:00:23,377 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609658] 2019-08-22 15:00:23,380 ReadCallback.java:242 - Digest mismatch:
DEBUG [PerDiskMemtableFlushWriter_1:9423] 2019-08-22 15:00:23,459 Memtable.java:456 - Writing Memtable-data@1393473686(12.552MiB serialized bytes, 76 ops, 0%/19% of on/off-heap limit), flushed range = (max(-1462500924879604616), max(3778679129973661054)]
DEBUG [PerDiskMemtableFlushWriter_0:9424] 2019-08-22 15:00:23,459 Memtable.java:456 - Writing Memtable-data@1393473686(12.552MiB serialized bytes, 76 ops, 0%/19% of on/off-heap limit), flushed range = (min(-9223372036854775808), max(-1462500924879604616)]
DEBUG [PerDiskMemtableFlushWriter_1:9423] 2019-08-22 15:00:23,464 Memtable.java:485 - Completed flushing /srv/sdb4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40017-big-Data.db (553.905KiB) for commitlog position CommitLogPosition(segmentId=1565021954787, position=24521324)
DEBUG [PerDiskMemtableFlushWriter_2:9424] 2019-08-22 15:00:23,465 Memtable.java:456 - Writing Memtable-data@1393473686(12.552MiB serialized bytes, 76 ops, 0%/19% of on/off-heap limit), flushed range = (max(3778679129973661054), max(9223372036854775807)]
INFO [ScheduledTasks:1] 2019-08-22 15:00:23,465 NoSpamLogger.java:91 - Some operations were slow, details available at debug level (debug.log)
DEBUG [ScheduledTasks:1] 2019-08-22 15:00:23,466 MonitoringTask.java:173 - 7 operations were slow in the last 4998 msecs:
DEBUG [COMMIT-LOG-ALLOCATOR] 2019-08-22 15:00:23,546 AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a fresh one
DEBUG [PerDiskMemtableFlushWriter_0:9424] 2019-08-22 15:00:23,621 Memtable.java:485 - Completed flushing /srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40016-big-Data.db (6.049MiB) for commitlog position CommitLogPosition(segmentId=1565021954787, position=24521324)
DEBUG [PerDiskMemtableFlushWriter_2:9424] 2019-08-22 15:00:23,648 Memtable.java:485 - Completed flushing /srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40018-big-Data.db (5.962MiB) for commitlog position CommitLogPosition(segmentId=1565021954787, position=24521324)
DEBUG [MemtableFlushWriter:9423] 2019-08-22 15:00:23,673 ColumnFamilyStore.java:1216 - Flushed to [BigTableReader(path='/srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40016-big-Data.db'), BigTableReader(path='/srv/sdb4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40017-big-Data.db'), BigTableReader(path='/srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40018-big-Data.db')] (3 sstables, 2.408MiB), biggest 1.140MiB, smallest 168.172KiB
DEBUG [ReadRepairStage:609628] 2019-08-22 15:00:23,739 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609629] 2019-08-22 15:00:24,060 ReadCallback.java:242 - Digest mismatch:
DEBUG [CompactionExecutor:45588] 2019-08-22 15:00:24,317 CompactionTask.java:255 - Compacted (8fd401c0-c4ed-11e9-ad4b-e11861b6015e) 4 sstables to [/srv/sdb4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40015-big,] to level=0. 18.402MiB to 16.405MiB (~89% of original) in 3,232ms. Read Throughput = 5.693MiB/s, Write Throughput = 5.075MiB/s, Row Throughput = ~378/s. 795 total partitions merged to 757. Partition merge counts were {1:724, 2:28, 3:5, }
DEBUG [ReadRepairStage:609660] 2019-08-22 15:00:24,548 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609630] 2019-08-22 15:00:24,555 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609628] 2019-08-22 15:00:24,868 ReadCallback.java:242 - Digest mismatch:
DEBUG [NativePoolCleaner] 2019-08-22 15:00:24,912 ColumnFamilyStore.java:1308 - Flushing largest CFS(Keyspace='enwiki_T_parsoid', ColumnFamily='data') to free up room. Used total: 0.01/0.50, live: 0.01/0.50, flushing: 0.00/0.00, this: 0.00/0.18
DEBUG [NativePoolCleaner] 2019-08-22 15:00:24,912 ColumnFamilyStore.java:918 - Enqueuing flush of data: 2.637KiB (0%) on-heap, 557.222MiB (18%) off-heap
DEBUG [COMMIT-LOG-ALLOCATOR] 2019-08-22 15:00:24,924 AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a fresh one
DEBUG [ReadRepairStage:609660] 2019-08-22 15:00:25,062 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609635] 2019-08-22 15:00:25,103 ReadCallback.java:242 - Digest mismatch:
DEBUG [PerDiskMemtableFlushWriter_2:9424] 2019-08-22 15:00:25,135 Memtable.java:456 - Writing Memtable-data@238293468(11.862MiB serialized bytes, 66 ops, 0%/19% of on/off-heap limit), flushed range = (max(3778679129973661054), max(9223372036854775807)]
DEBUG [PerDiskMemtableFlushWriter_0:9424] 2019-08-22 15:00:25,135 Memtable.java:456 - Writing Memtable-data@238293468(11.862MiB serialized bytes, 66 ops, 0%/19% of on/off-heap limit), flushed range = (min(-9223372036854775808), max(-1462500924879604616)]
DEBUG [PerDiskMemtableFlushWriter_1:9423] 2019-08-22 15:00:25,142 Memtable.java:456 - Writing Memtable-data@238293468(11.862MiB serialized bytes, 66 ops, 0%/19% of on/off-heap limit), flushed range = (max(-1462500924879604616), max(3778679129973661054)]
DEBUG [PerDiskMemtableFlushWriter_1:9423] 2019-08-22 15:00:25,143 Memtable.java:485 - Completed flushing /srv/sdb4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40020-big-Data.db (61.480KiB) for commitlog position CommitLogPosition(segmentId=1565021954788, position=31174485)
DEBUG [CompactionExecutor:45589] 2019-08-22 15:00:25,163 CompactionTask.java:255 - Compacted (8ee4bfc0-c4ed-11e9-ad4b-e11861b6015e) 4 sstables to [/srv/sda4/cassandra-b/data/wikipedia_T_parsoid/data-db05bba065db11e9bc6c8958654eb0a6/mc-49097-big,] to level=0. 70.042MiB to 62.930MiB (~89% of original) in 5,645ms. Read Throughput = 12.407MiB/s, Write Throughput = 11.147MiB/s, Row Throughput = ~531/s. 1,702 total partitions merged to 1,595. Partition merge counts were {1:1508, 2:70, 3:14, 4:3, }
DEBUG [ReadRepairStage:609636] 2019-08-22 15:00:25,202 ReadCallback.java:242 - Digest mismatch:
DEBUG [PerDiskMemtableFlushWriter_2:9424] 2019-08-22 15:00:25,233 Memtable.java:485 - Completed flushing /srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40021-big-Data.db (5.979MiB) for commitlog position CommitLogPosition(segmentId=1565021954788, position=31174485)
DEBUG [PerDiskMemtableFlushWriter_0:9424] 2019-08-22 15:00:25,247 Memtable.java:485 - Completed flushing /srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40019-big-Data.db (5.823MiB) for commitlog position CommitLogPosition(segmentId=1565021954788, position=31174485)
DEBUG [ReadRepairStage:609647] 2019-08-22 15:00:25,248 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609637] 2019-08-22 15:00:25,321 ReadCallback.java:242 - Digest mismatch:
DEBUG [MemtableFlushWriter:9423] 2019-08-22 15:00:25,350 ColumnFamilyStore.java:1216 - Flushed to [BigTableReader(path='/srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40019-big-Data.db'), BigTableReader(path='/srv/sdb4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40020-big-Data.db'), BigTableReader(path='/srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40021-big-Data.db')] (3 sstables, 2.197MiB), biggest 1.110MiB, smallest 19.190KiB
DEBUG [CompactionExecutor:45582] 2019-08-22 15:00:25,350 CompactionTask.java:155 - Compacting (925ef260-c4ed-11e9-ad4b-e11861b6015e) [/srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40012-big-Data.db:level=0, /srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40019-big-Data.db:level=0, /srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40008-big-Data.db:level=0, /srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40016-big-Data.db:level=0, ]
DEBUG [ReadRepairStage:609651] 2019-08-22 15:00:25,355 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609663] 2019-08-22 15:00:25,489 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609662] 2019-08-22 15:00:25,686 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609654] 2019-08-22 15:00:25,730 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609632] 2019-08-22 15:00:25,784 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609631] 2019-08-22 15:00:25,796 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609627] 2019-08-22 15:00:25,801 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609664] 2019-08-22 15:00:25,817 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609625] 2019-08-22 15:00:25,828 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609633] 2019-08-22 15:00:25,842 ReadCallback.java:242 - Digest mismatch:
DEBUG [COMMIT-LOG-ALLOCATOR] 2019-08-22 15:00:25,918 AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a fresh one
DEBUG [ReadRepairStage:609634] 2019-08-22 15:00:26,063 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609655] 2019-08-22 15:00:26,220 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609630] 2019-08-22 15:00:26,230 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609642] 2019-08-22 15:00:26,283 ReadCallback.java:242 - Digest mismatch:
DEBUG [NativePoolCleaner] 2019-08-22 15:00:26,333 ColumnFamilyStore.java:1308 - Flushing largest CFS(Keyspace='enwiki_T_parsoid', ColumnFamily='data') to free up room. Used total: 0.01/0.68, live: 0.01/0.49, flushing: 0.00/0.18, this: 0.00/0.17
DEBUG [NativePoolCleaner] 2019-08-22 15:00:26,333 ColumnFamilyStore.java:918 - Enqueuing flush of data: 1.172KiB (0%) on-heap, 512.418MiB (17%) off-heap
DEBUG [COMMIT-LOG-ALLOCATOR] 2019-08-22 15:00:26,336 AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a fresh one
DEBUG [PerDiskMemtableFlushWriter_1:9423] 2019-08-22 15:00:26,387 Memtable.java:456 - Writing Memtable-data@1531469497(11.764MiB serialized bytes, 51 ops, 0%/17% of on/off-heap limit), flushed range = (max(-1462500924879604616), max(3778679129973661054)]
DEBUG [PerDiskMemtableFlushWriter_0:9424] 2019-08-22 15:00:26,387 Memtable.java:456 - Writing Memtable-data@1531469497(11.764MiB serialized bytes, 51 ops, 0%/17% of on/off-heap limit), flushed range = (min(-9223372036854775808), max(-1462500924879604616)]
DEBUG [PerDiskMemtableFlushWriter_1:9423] 2019-08-22 15:00:26,387 Memtable.java:485 - Completed flushing /srv/sdb4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40024-big-Data.db (0.000KiB) for commitlog position CommitLogPosition(segmentId=1565021954790, position=30804115)
DEBUG [PerDiskMemtableFlushWriter_2:9424] 2019-08-22 15:00:26,388 Memtable.java:456 - Writing Memtable-data@1531469497(11.764MiB serialized bytes, 51 ops, 0%/17% of on/off-heap limit), flushed range = (max(3778679129973661054), max(9223372036854775807)]
DEBUG [ReadRepairStage:609649] 2019-08-22 15:00:26,449 ReadCallback.java:242 - Digest mismatch:
DEBUG [ReadRepairStage:609659] 2019-08-22 15:00:26,478 ReadCallback.java:242 - Digest mismatch:
DEBUG [PerDiskMemtableFlushWriter_2:9424] 2019-08-22 15:00:26,501 Memtable.java:485 - Completed flushing /srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40025-big-Data.db (5.926MiB) for commitlog position CommitLogPosition(segmentId=1565021954790, position=30804115)
DEBUG [PerDiskMemtableFlushWriter_0:9424] 2019-08-22 15:00:26,519 Memtable.java:485 - Completed flushing /srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40023-big-Data.db (5.839MiB) for commitlog position CommitLogPosition(segmentId=1565021954790, position=30804115)
DEBUG [MemtableFlushWriter:9423] 2019-08-22 15:00:26,555 ColumnFamilyStore.java:1216 - Flushed to [BigTableReader(path='/srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40023-big-Data.db'), BigTableReader(path='/srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40025-big-Data.db')] (2 sstables, 2.167MiB), biggest 1.094MiB, smallest 1.074MiB
DEBUG [CompactionExecutor:45584] 2019-08-22 15:00:26,556 CompactionTask.java:155 - Compacting (9316d0b0-c4ed-11e9-ad4b-e11861b6015e) [/srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40018-big-Data.db:level=0, /srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40014-big-Data.db:level=0, /srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40021-big-Data.db:level=0, /srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40025-big-Data.db:level=0, ]
DEBUG [CompactionExecutor:45582] 2019-08-22 15:00:26,602 CompactionTask.java:255 - Compacted (925ef260-c4ed-11e9-ad4b-e11861b6015e) 4 sstables to [/srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40022-big,] to level=0. 9.376MiB to 6.167MiB (~65% of original) in 1,235ms. Read Throughput = 7.589MiB/s, Write Throughput = 4.991MiB/s, Row Throughput = ~181/s. 185 total partitions merged to 181. Partition merge counts were {1:179, 2:1, 4:1, }
DEBUG [ReadRepairStage:609656] 2019-08-22 15:00:26,731 ReadCallback.java:242 - Digest mismatch:
DEBUG [COMMIT-LOG-ALLOCATOR] 2019-08-22 15:00:27,177 AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a fresh one
DEBUG [NativePoolCleaner] 2019-08-22 15:00:27,293 ColumnFamilyStore.java:1308 - Flushing largest CFS(Keyspace='enwiki_T_parsoid', ColumnFamily='data') to free up room. Used total: 0.01/0.67, live: 0.01/0.50, flushing: 0.00/0.17, this: 0.00/0.18
DEBUG [NativePoolCleaner] 2019-08-22 15:00:27,293 ColumnFamilyStore.java:918 - Enqueuing flush of data: 0.586KiB (0%) on-heap, 548.402MiB (18%) off-heap
DEBUG [CompactionExecutor:45584] 2019-08-22 15:00:27,388 CompactionTask.java:255 - Compacted (9316d0b0-c4ed-11e9-ad4b-e11861b6015e) 4 sstables to [/srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40026-big,] to level=0. 4.529MiB to 1.278MiB (~28% of original) in 831ms. Read Throughput = 5.446MiB/s, Write Throughput = 1.537MiB/s, Row Throughput = ~44/s. 25 total partitions merged to 22. Partition merge counts were {1:21, 4:1, }
DEBUG [PerDiskMemtableFlushWriter_0:9424] 2019-08-22 15:00:27,516 Memtable.java:456 - Writing Memtable-data@1832774074(11.723MiB serialized bytes, 40 ops, 0%/20% of on/off-heap limit), flushed range = (min(-9223372036854775808), max(-1462500924879604616)]
DEBUG [PerDiskMemtableFlushWriter_1:9423] 2019-08-22 15:00:27,522 Memtable.java:456 - Writing Memtable-data@1832774074(11.723MiB serialized bytes, 40 ops, 0%/20% of on/off-heap limit), flushed range = (max(-1462500924879604616), max(3778679129973661054)]
DEBUG [PerDiskMemtableFlushWriter_1:9423] 2019-08-22 15:00:27,523 Memtable.java:485 - Completed flushing /srv/sdb4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40028-big-Data.db (0.000KiB) for commitlog position CommitLogPosition(segmentId=1565021954792, position=6102662)
DEBUG [PerDiskMemtableFlushWriter_2:9424] 2019-08-22 15:00:27,523 Memtable.java:456 - Writing Memtable-data@1832774074(11.723MiB serialized bytes, 40 ops, 0%/20% of on/off-heap limit), flushed range = (max(3778679129973661054), max(9223372036854775807)]
INFO [Service Thread] 2019-08-22 15:00:27,828 GCInspector.java:284 - G1 Young Generation GC in 216ms. G1 Eden Space: 469762048 -> 0; G1 Old Gen: 10564281752 -> 8725110808;
DEBUG [COMMIT-LOG-ALLOCATOR] 2019-08-22 15:00:27,851 AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a fresh one
DEBUG [PerDiskMemtableFlushWriter_2:9424] 2019-08-22 15:00:27,901 Memtable.java:485 - Completed flushing /srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40029-big-Data.db (5.909MiB) for commitlog position CommitLogPosition(segmentId=1565021954792, position=6102662)
DEBUG [PerDiskMemtableFlushWriter_0:9424] 2019-08-22 15:00:27,908 Memtable.java:485 - Completed flushing /srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40027-big-Data.db (5.814MiB) for commitlog position CommitLogPosition(segmentId=1565021954792, position=6102662)
DEBUG [MemtableFlushWriter:9423] 2019-08-22 15:00:27,923 ColumnFamilyStore.java:1216 - Flushed to [BigTableReader(path='/srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40027-big-Data.db'), BigTableReader(path='/srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40029-big-Data.db')] (2 sstables, 2.154MiB), biggest 1.088MiB, smallest 1.066MiB
DEBUG [NativePoolCleaner] 2019-08-22 15:00:28,175 ColumnFamilyStore.java:1308 - Flushing largest CFS(Keyspace='enwiki_T_parsoid', ColumnFamily='data') to free up room. Used total: 0.01/0.68, live: 0.01/0.48, flushing: 0.00/0.18, this: 0.00/0.15
DEBUG [NativePoolCleaner] 2019-08-22 15:00:28,175 ColumnFamilyStore.java:918 - Enqueuing flush of data: 0.586KiB (0%) on-heap, 463.869MiB (15%) off-heap
DEBUG [ReadRepairStage:609634] 2019-08-22 15:00:28,179 ReadCallback.java:242 - Digest mismatch:
DEBUG [Native-Transport-Requests-22] 2019-08-22 15:00:29,909 ReadCallback.java:132 - Timed out; received 2 of 3 responses (including data)
WARN [Service Thread] 2019-08-22 15:00:29,909 GCInspector.java:282 - G1 Young Generation GC in 1639ms. G1 Eden Space: 461373440 -> 0; G1 Old Gen: 10667456672 -> 9641073664;
DEBUG [ScheduledTasks:1] 2019-08-22 15:00:29,912 MonitoringTask.java:173 - 185 operations were slow in the last 4806 msecs:
DEBUG [Native-Transport-Requests-17] 2019-08-22 15:00:29,977 DataResolver.java:200 - Timeout while read-repairing after receiving all 6 data and digest responses
DEBUG [Native-Transport-Requests-71] 2019-08-22 15:00:29,977 ReadCallback.java:132 - Timed out; received 2 of 3 responses (including data)
DEBUG [Native-Transport-Requests-54] 2019-08-22 15:00:29,977 ReadCallback.java:132 - Timed out; received 2 of 3 responses (including data)
DEBUG [Native-Transport-Requests-49] 2019-08-22 15:00:29,977 ReadCallback.java:132 - Timed out; received 2 of 3 responses (including data)
DEBUG [ReadRepairStage:609648] 2019-08-22 15:00:29,984 DataResolver.java:200 - Timeout while read-repairing after receiving all 2 data and digest responses
DEBUG [ReadRepairStage:609645] 2019-08-22 15:00:29,986 DataResolver.java:200 - Timeout while read-repairing after receiving all 2 data and digest responses
ERROR [ReadRepairStage:609648] 2019-08-22 15:00:29,999 CassandraDaemon.java:228 - Exception in thread Thread[ReadRepairStage:609648,5,main]
DEBUG [Native-Transport-Requests-6] 2019-08-22 15:00:30,019 ReadCallback.java:132 - Timed out; received 2 of 3 responses (including data)
ERROR [ReadRepairStage:609645] 2019-08-22 15:00:30,020 CassandraDaemon.java:228 - Exception in thread Thread[ReadRepairStage:609645,5,main]
DEBUG [Native-Transport-Requests-41] 2019-08-22 15:00:30,021 DataResolver.java:200 - Timeout while read-repairing after receiving all 6 data and digest responses
DEBUG [Native-Transport-Requests-3] 2019-08-22 15:00:30,020 ReadCallback.java:132 - Timed out; received 1 of 3 responses (including data)
DEBUG [COMMIT-LOG-ALLOCATOR] 2019-08-22 15:00:30,039 AbstractCommitLogSegmentManager.java:109 - No segments in reserve; creating a fresh one
DEBUG [Native-Transport-Requests-8] 2019-08-22 15:00:30,088 ReadCallback.java:132 - Timed out; received 2 of 3 responses (including data)
DEBUG [PerDiskMemtableFlushWriter_0:9424] 2019-08-22 15:00:30,117 Memtable.java:456 - Writing Memtable-data@929332878(11.723MiB serialized bytes, 39 ops, 0%/20% of on/off-heap limit), flushed range = (min(-9223372036854775808), max(-1462500924879604616)]
DEBUG [PerDiskMemtableFlushWriter_1:9423] 2019-08-22 15:00:30,122 Memtable.java:456 - Writing Memtable-data@929332878(11.723MiB serialized bytes, 39 ops, 0%/20% of on/off-heap limit), flushed range = (max(-1462500924879604616), max(3778679129973661054)]
DEBUG [PerDiskMemtableFlushWriter_1:9423] 2019-08-22 15:00:30,123 Memtable.java:485 - Completed flushing /srv/sdb4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40031-big-Data.db (0.000KiB) for commitlog position CommitLogPosition(segmentId=1565021954793, position=18598938)
DEBUG [PerDiskMemtableFlushWriter_2:9424] 2019-08-22 15:00:30,123 Memtable.java:456 - Writing Memtable-data@929332878(11.723MiB serialized bytes, 39 ops, 0%/20% of on/off-heap limit), flushed range = (max(3778679129973661054), max(9223372036854775807)]
DEBUG [Native-Transport-Requests-66] 2019-08-22 15:00:30,161 ReadCallback.java:132 - Timed out; received 1 of 3 responses (including data)
DEBUG [PerDiskMemtableFlushWriter_2:9424] 2019-08-22 15:00:30,230 Memtable.java:485 - Completed flushing /srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40032-big-Data.db (5.909MiB) for commitlog position CommitLogPosition(segmentId=1565021954793, position=18598938)
DEBUG [Native-Transport-Requests-4] 2019-08-22 15:00:30,533 ReadCallback.java:132 - Timed out; received 1 of 3 responses (including data)
DEBUG [ReadRepairStage:609664] 2019-08-22 15:00:30,534 ReadCallback.java:242 - Digest mismatch:
DEBUG [PerDiskMemtableFlushWriter_0:9424] 2019-08-22 15:00:30,565 Memtable.java:485 - Completed flushing /srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40030-big-Data.db (5.814MiB) for commitlog position CommitLogPosition(segmentId=1565021954793, position=18598938)
DEBUG [MemtableFlushWriter:9423] 2019-08-22 15:00:30,587 ColumnFamilyStore.java:1216 - Flushed to [BigTableReader(path='/srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40030-big-Data.db'), BigTableReader(path='/srv/sdc4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40032-big-Data.db')] (2 sstables, 2.154MiB), biggest 1.088MiB, smallest 1.066MiB
DEBUG [CompactionExecutor:45581] 2019-08-22 15:00:30,587 CompactionTask.java:155 - Compacting (957e0cb0-c4ed-11e9-ad4b-e11861b6015e) [/srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40022-big-Data.db:level=0, /srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40023-big-Data.db:level=0, /srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40027-big-Data.db:level=0, /srv/sda4/cassandra-b/data/enwiki_T_parsoid/data-ae7fe34065df11e9bc6c8958654eb0a6/mc-40030-big-Data.db:level=0, ]
DEBUG [Native-Transport-Requests-26] 2019-08-22 15:00:30,907 ReadCallback.java:132 - Timed out; received 2 of 3 responses (including data)
DEBUG [ReadRepairStage:609650] 2019-08-22 15:00:31,011 ReadCallback.java:242 - Digest mismatch:
DEBUG [NativePoolCleaner] 2019-08-22 15:00:31,118 ColumnFamilyStore.java:1308 - Flushing largest CFS(Keyspace='enwiki_T_parsoid', ColumnFamily='data') to free up room. Used total: 0.01/0.65, live: 0.01/0.46, flushing: 0.00/0.15, this: 0.00/0.12
DEBUG [NativePoolCleaner] 2019-08-22 15:00:31,118 ColumnFamilyStore.java:918 - Enqueuing flush of data: 0.586KiB (0%) on-heap, 376.564MiB (12%) off-heap

Event Timeline

Eevans changed the title of this paste from untitled to Masterwork From Distant Lands.
Eevans changed the title of this paste from Masterwork From Distant Lands to /var/log/cassandra/debug-b.log.