Page MenuHomePhabricator

Convert RESTBase Cassandra cluster to TimeWindowCompactionStrategy
Closed, ResolvedPublic

Description

DateTieredCompactionStrategy (DTCS) is not working as expected, its optimizations are being defeated in our environment(s) by out-of-order writes (see T126221: Evaluate efficacy of DateTieredCompactionStrategy, for background). An alternative to DTCS has emerged in the form of TimeWindowCompactionStrategy (TWCS), which eschews tiering in favor of creating fixed windows of time.

Since time-ordered data models are common in our environment(s), I believe TWCS warrants an investigation.

Status

Tables that have been converted to date.

Conversion dateTables
2016-10-12local_group_wiktionary_T_parsoid_html.dataSSTables/read, SSTable count (large spikes are the result of repair testing)
2016-10-13local_group_wikimedia_T_parsoid_html.dataSSTables/read, SSTable count
2016-10-19local_group_wikipedia_T_parsoid_html.dataSSTables/read. SSTable count
2016-10-27local_group_*_T_mobileapps_{lead,remaining}.data
2016-11-07local_group_*_T_title__revisions.{data,idx_by_rev_ever}

Tombstone GC

One of the primary hopes for TWCS, was that when combined with repair, out-of-order writes could be confined to the STCS-compacted current window. For repairs completed within the compaction_window_size, overlap could be largely eliminated before the outgoing window's major compaction, and droppable tombstones thus kept to a minimum. That may continue to be an option for some subset of tables, but testing conducted as part of T113805 would suggest it is unlikely that we have the overhead to complete frequent repairs of more than a subset of our data.

However, starting in the comments here, are the results of user-defined compaction tests that collapse the N oldest windows, resulting in significant collection of tombstones. User-defined compactions of this kind would be straightforward to script, are not impactful, and have the added benefit of bounding the number of SSTables (and as a result, the SSTables/read).

Update: 2016-11-29

A proof-of-concept script has been put in place, running out of my (@Eevans) crontab, that performs the user-defined compactions described above (as a limited trial). We should revisit this in a month or so, and if it continues to produce favorable results (and no better alternative presents itself in the meantime) then we can properly operationalize this.


Details

Related Gerrit Patches:
operations/puppet : productionAdd time-window compaction strategy jar to classpath
operations/puppet : productionEnable cassandra/twcs deploy repository
operations/puppet : productionExtend classpath via Puppet
operations/software/cassandra-twcs : masterCassandra TWCS deploy repository

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 22 2016, 3:26 PM
Eevans triaged this task as Lowest priority.Apr 22 2016, 3:26 PM
Eevans moved this task from Backlog to Next on the Cassandra board.Sep 12 2016, 7:36 PM

Mentioned in SAL (#wikimedia-operations) [2016-09-22T20:20:50Z] <urandom> T133395: RESTBase Staging: Restarting Cassandra to pick up TWCS jar in classpath

Mentioned in SAL (#wikimedia-operations) [2016-09-22T20:47:38Z] <urandom> T133395: RESTBase Staging: altering table to set TWCS on wikipedia parsoid.html table

Mentioned in SAL (#wikimedia-operations) [2016-09-22T20:52:05Z] <urandom> T133395: RESTBase Staging: starting dumps (3, eqiad)

A request for the deploy repository (operations/software/cassandra-twcs) has been submitted (see https://www.mediawiki.org/wiki/Git/New_repositories/Requests).

Change 313619 had a related patch set uploaded (by Eevans):
Extend classpath via Puppet

https://gerrit.wikimedia.org/r/313619

Eevans moved this task from Next to In-Progress on the Cassandra board.Oct 3 2016, 3:47 PM

Change 313825 had a related patch set uploaded (by Eevans):
Adding TimeWindowCompactionStrategy-2.2.5.jar via git-fat

https://gerrit.wikimedia.org/r/313825

Change 313892 had a related patch set uploaded (by Eevans):
Enable cassandra/twcs deploy repository

https://gerrit.wikimedia.org/r/313892

Change 313825 merged by Eevans:
Cassandra TWCS deploy repository

https://gerrit.wikimedia.org/r/313825

Change 313619 merged by Filippo Giunchedi:
Extend classpath via Puppet

https://gerrit.wikimedia.org/r/313619

Change 313892 merged by Filippo Giunchedi:
Enable cassandra/twcs deploy repository

https://gerrit.wikimedia.org/r/313892

Eevans added a comment.Oct 6 2016, 8:54 PM

The jar file is now deployed to production nodes (deployment setup documented here). We can enable this (by appending the jar to the classpath), as soon as Gerrit is back up.

Change 314603 had a related patch set uploaded (by Eevans):
Add time-window compaction strategy jar to classpath

https://gerrit.wikimedia.org/r/314603

Change 314603 merged by Elukey:
Add time-window compaction strategy jar to classpath

https://gerrit.wikimedia.org/r/314603

Mentioned in SAL (#wikimedia-operations) [2016-10-11T14:20:01Z] <urandom> T133395: Restarting xenon.eqiad.wmnet to apply https://gerrit.wikimedia.org/r/314603

A small hiccup while applying the proper jar deploy/classpath configuration to Staging (xenon and praseodymium have incomplete/broken scap deploys of the strategy jar).

$ cdsh -c restbase-test -- file /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar
praseodymium.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: ASCII text
cerium.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
xenon.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: ASCII text
restbase-test2001.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase-test2002.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase-test2003.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract

Production is OK:

$ cdsh -- file /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar
restbase1007.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase1010.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase1011.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase1008.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase1012.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase1013.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase1009.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase1014.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase1015.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase2003.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase2004.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase2008.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase2001.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase2002.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase2007.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase2005.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase2006.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase2009.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract

A small hiccup while applying the proper jar deploy/classpath configuration to Staging (xenon and praseodymium have incomplete/broken scap deploys of the strategy jar).

$ cdsh -c restbase-test -- file /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar
praseodymium.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: ASCII text
cerium.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
xenon.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: ASCII text
restbase-test2001.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase-test2002.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase-test2003.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract

"Fixed" by issuing cd /srv/deployment/cassandra/twcs && sudo -u deploy-service git fat pull from the affected hosts.

$ cdsh -c restbase-test -- file /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar
praseodymium.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
cerium.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
xenon.eqiad.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase-test2001.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase-test2002.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract
restbase-test2003.codfw.wmnet: /srv/deployment/cassandra/twcs/lib/cassandra-v2.2/TimeWindowCompactionStrategy-2.2.5.jar: Zip archive data, at least v1.0 to extract

Mentioned in SAL (#wikimedia-operations) [2016-10-11T18:09:57Z] <urandom> T133395: Restarting xenon.eqiad.wmnet to apply https://gerrit.wikimedia.org/r/314603

Mentioned in SAL (#wikimedia-operations) [2016-10-11T18:13:34Z] <urandom> T133395: Restarting Cassandra in RESTBase Staging to apply https://gerrit.wikimedia.org/r/314603

Mentioned in SAL (#wikimedia-operations) [2016-10-11T18:26:38Z] <urandom> T133395: Starting dumps (3) in RESTBase Staging

Mentioned in SAL (#wikimedia-operations) [2016-10-11T18:33:44Z] <urandom> T133395: Restarting Cassandra in RESTBase (codfw) to apply https://gerrit.wikimedia.org/r/314603

Mentioned in SAL (#wikimedia-operations) [2016-10-11T20:03:15Z] <urandom> T133395: Restarting Cassandra: restbase2008-c

Mentioned in SAL (#wikimedia-operations) [2016-10-11T20:58:42Z] <urandom> T133395: Restart Cassandra on restbase2005-a.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2016-10-11T21:06:20Z] <urandom> T133395: Restart Cassandra on restbase2005-b.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2016-10-11T21:10:19Z] <urandom> T133395: Restart Cassandra on restbase2005-c.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2016-10-11T21:17:08Z] <urandom> T133395: Restarting Cassandra instances on restbase2006.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2016-10-12T14:04:10Z] <urandom> T133395: Restarting Cassandra instances on restbase2009.codfw.wmnet

Codfw instance restarts are complete; TimeWindowCompactionStrategy has been added to the classpath:

$ cdsh -d codfw -- pgrep -ac -f 'TimeWindowCompactionStrategy-2.2.5.jar.+CassandraDaemon'
restbase2003.codfw.wmnet: 3
restbase2004.codfw.wmnet: 3
restbase2008.codfw.wmnet: 3
restbase2001.codfw.wmnet: 3
restbase2002.codfw.wmnet: 3
restbase2007.codfw.wmnet: 3
restbase2005.codfw.wmnet: 3
restbase2006.codfw.wmnet: 3
restbase2009.codfw.wmnet: 3

Mentioned in SAL (#wikimedia-operations) [2016-10-12T14:22:06Z] <urandom> T133395: Restarting Cassandra instances on restbase1007.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2016-10-12T14:34:38Z] <urandom> T133395: Restarting Cassandra instances on restbase1010.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2016-10-12T14:50:58Z] <urandom> T133395: Restarting Cassandra instances on restbase1011.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2016-10-12T15:10:26Z] <urandom> T133395: Restarting Cassandra instances in RESTBase, eqiad, rack 'b'

Mentioned in SAL (#wikimedia-operations) [2016-10-12T15:51:00Z] <urandom> T133395: Restarting Cassandra instances in RESTBase, eqiad, rack 'd'

Eqiad instance restarts are complete; TimeWindowCompactionStrategy is present on the classpath:

$ cdsh -d eqiad -- pgrep -c -f 'TimeWindowCompactionStrategy-2.2.5.jar.+org.apache.cassandra.service.CassandraDaemon'
restbase1007.eqiad.wmnet: 3
restbase1010.eqiad.wmnet: 3
restbase1011.eqiad.wmnet: 3
restbase1008.eqiad.wmnet: 3
restbase1012.eqiad.wmnet: 3
restbase1013.eqiad.wmnet: 3
restbase1009.eqiad.wmnet: 3
restbase1014.eqiad.wmnet: 3
restbase1015.eqiad.wmnet: 3

Proposed ALTERs to kick things off. First on Wiktionary parsoid.html for a confidence-building period, then Wikimedia:

1-- Wiktionary
2ALTER TABLE
3 "local_group_wiktionary_T_parsoid_html".data
4WITH
5 compaction= {
6 'compaction_window_unit': 'DAYS',
7 'compaction_window_size': '14',
8 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy'
9 };
10
11-- Wikimedia (Commons)
12ALTER TABLE
13 "local_group_wikimedia_T_parsoid_html".data
14WITH
15 compaction= {
16 'compaction_window_unit': 'DAYS',
17 'compaction_window_size': '14',
18 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy'
19 };

Mentioned in SAL (#wikimedia-operations) [2016-10-12T17:36:08Z] <urandom> T133395: RESTBase: Altering keyspace local_group_wiktionary_T_parsoid_html.data to enable time-window compaction

Mentioned in SAL (#wikimedia-operations) [2016-10-13T15:26:06Z] <urandom> T133395: RESTBase: Altering keyspace local_group_wikimedia_T_parsoid_html.data to enable time-window compaction

Mentioned in SAL (#wikimedia-operations) [2016-10-19T18:02:42Z] <urandom> T133395: RESTBase: Altering keyspace local_group_wikipedia_T_parsoid_html.data to enable time-window compaction

Mentioned in SAL (#wikimedia-operations) [2016-10-20T17:42:56Z] <urandom> T133395, T113805: Starting a primary-range, incremental repair of local_group_wiktionary_T_parsoid_html.data on restbase2001.codfw.wmnet

Proposed ALTERs for mobileapps keyspaces:

1ALTER TABLE "local_group_wikiquote_T_mobileapps_lead".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
2ALTER TABLE "local_group_wikiquote_T_mobileapps_remaining".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
3ALTER TABLE "local_group_wikisource_T_mobileapps_remaining".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
4ALTER TABLE "local_group_phase0_T_mobileapps_lead".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
5ALTER TABLE "local_group_default_T_mobileapps_lead".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
6ALTER TABLE "local_group_wikinews_T_mobileapps_remaining".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
7ALTER TABLE "local_group_phase0_T_mobileapps_remaining".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
8ALTER TABLE "local_group_wikipedia_T_mobileapps_remaining".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
9ALTER TABLE "local_group_wikiversity_T_mobileapps_lead".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
10ALTER TABLE "local_group_wikibooks_T_mobileapps_lead".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
11ALTER TABLE "local_group_wikimedia_T_mobileapps_remaining".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
12ALTER TABLE "local_group_default_T_mobileapps_remaining".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
13ALTER TABLE "local_group_wiktionary_T_mobileapps_remaining".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
14ALTER TABLE "local_group_wikimedia_T_mobileapps_lead".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
15ALTER TABLE "local_group_wiktionary_T_mobileapps_lead".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
16ALTER TABLE "local_group_wikivoyage_T_mobileapps_lead".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
17ALTER TABLE "local_group_wikipedia_T_mobileapps_lead".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
18ALTER TABLE "local_group_wikinews_T_mobileapps_lead".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
19ALTER TABLE "local_group_wikibooks_T_mobileapps_remaining".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
20ALTER TABLE "local_group_wikisource_T_mobileapps_lead".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
21ALTER TABLE "local_group_wikiversity_T_mobileapps_remaining".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };
22ALTER TABLE "local_group_wikivoyage_T_mobileapps_remaining".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };

Eevans raised the priority of this task from Lowest to Medium.Oct 27 2016, 7:37 PM
Eevans edited projects, added Services (doing); removed Services, RESTBase-Cassandra.

Mentioned in SAL (#wikimedia-operations) [2016-10-27T21:10:48Z] <urandom> T133395: Altering mobileapps keyspaces to use time-windowed compaction

Eevans added a comment.Nov 3 2016, 6:21 PM

Looking at the SSTables of local_group_wikipedia_T_parsoid_html.data on 2001-a yesterday you can see that there are two windows, one with a lower-bound of 2016-10-06, and one with a lower-bound of 2016-10-20. The single table in the oldest window has a droppable tombstone ratio of 0 (expected after what would have amounted to a node-wide major compaction).

1$ sudo ./inspect /srv/cassandra-a/data/local_group_wikipedia_T_parsoid_html/data-f3648bc0c2cb11e49ce6a1da77f2fd34/
2 File Size Tombstones Repaired at
32016-10-06
4 la-169018-big-Data.db 254.9 GB 0.00% 0
5
62016-10-20
7 la-171205-big-Data.db 28.1 MB 52.12% 0
8 la-171206-big-Data.db 19.9 MB 48.73% 0
9 la-170097-big-Data.db 8.8 GB 89.80% 0
10 la-170844-big-Data.db 4.2 GB 84.59% 0
11 la-171204-big-Data.db 28 MB 59.75% 0
12 la-170439-big-Data.db 4.1 GB 72.56% 0
13 la-171149-big-Data.db 502 MB 62.48% 0
14 la-171193-big-Data.db 329.7 MB 48.24% 0
15 la-171203-big-Data.db 74.8 MB 56.68% 0
16 la-171198-big-Data.db 67.7 MB 52.59% 0
17 la-171172-big-Data.db 4.1 GB 70.96% 0
18
19
20Total size: 277.1 GB (data files only)

Today marks the boundary of a new 14 day window. Here you can see that the previous window (the one with a lower-bound starting on the 20th), has been compacted down to a single table. This table has 83% droppable tombstones due to the overlap created by a new window.

1 File Size Tombstones Repaired at
22016-10-06
3 la-169018-big-Data.db 254.9 GB 0.00% 0
4
52016-10-20
6 la-171281-big-Data.db 17.3 GB 83.06% 0
7
82016-11-03
9 la-171395-big-Data.db 21.5 MB 54.30% 0
10 la-171396-big-Data.db 20.9 MB 56.55% 0
11 la-171383-big-Data.db 408 MB 56.65% 0
12 la-171362-big-Data.db 1.3 GB 59.27% 0
13 la-171394-big-Data.db 23.3 MB 53.13% 0
14 la-171388-big-Data.db 92.2 MB 57.30% 0
15 la-171393-big-Data.db 79.6 MB 53.90% 0
16
17
18Total size: 274.1 GB (data files only)

NOTE: The local_group_wikipedia_T_parsoid_html.data table was converted to TWCS on 2016-10-19.
NOTE: I am tempted to perform a user-defined compaction of la-169018-big-Data.db and la-171281-big-Data.db to gauge the effect on droppable tombstones.

Mentioned in SAL (#wikimedia-operations) [2016-11-03T20:33:43Z] <urandom> T133395: Enabling unchecked_tombstone_compaction and setting tombstone_threshold = .6 on "local_group_wikipedia_T_parsoid_html".data

Eevans added a comment.Nov 4 2016, 8:58 PM

As an experiment, I enabled unchecked_tombstone_compaction, and the table in the 2016-10-20 has since been rewritten.

1 File Size Tombstones Repaired at
22016-10-06
3 la-169018-big-Data.db 254.9 GB 0.00% 0
4
52016-10-20
6 la-171488-big-Data.db 16.5 GB 83.08% 0
7
82016-11-03
9 la-171555-big-Data.db 262.2 MB 53.44% 0
10 la-171534-big-Data.db 1.7 GB 59.11% 0
11 la-171556-big-Data.db 21.1 MB 58.34% 0
12 la-171362-big-Data.db 1.3 GB 83.04% 0
13 la-171448-big-Data.db 1.6 GB 56.44% 0
14 la-171557-big-Data.db 34.5 MB 57.32% 0
15
16
17Total size: 276.3 GB (data files only)

The resulting file (generation no. 171488) didn't result in any significant tombstone GC, despite it being more than 24 hours old (the retention policy TTL), so this is indeed caused by overlap.

Mentioned in SAL (#wikimedia-operations) [2016-11-04T21:14:03Z] <urandom> T133395: Starting user-defined compaction of local_group_wikipedia_T_parsoid_html.data, files la-169018-big-Data.db and la-171488-big-Data.db

Eevans added a comment.Nov 7 2016, 4:39 PM

Aha, interesting! After a user-defined compaction of la-169018-big-Data.db and la-171488-big-Data.db above, the newly created table (la-171621-big-Data.db) has a droppable tombstone ratio of 0.

1 File Size Tombstones Repaired at
22016-10-20
3 la-171621-big-Data.db 264.8 GB 0.00% 0
4
52016-11-03
6 la-172191-big-Data.db 79.1 MB 56.36% 0
7 la-172201-big-Data.db 82.4 MB 56.47% 0
8 la-172211-big-Data.db 128.7 MB 61.08% 0
9 la-172212-big-Data.db 19.6 MB 56.15% 0
10 la-172064-big-Data.db 3.4 GB 83.23% 0
11 la-172186-big-Data.db 474.5 MB 57.45% 0
12 la-172165-big-Data.db 437 MB 58.92% 0
13 la-172144-big-Data.db 1.8 GB 64.66% 0
14 la-171986-big-Data.db 5.3 GB 79.19% 0
15 la-172214-big-Data.db 32.9 MB 47.74% 0
16 la-172196-big-Data.db 76 MB 54.23% 0
17 la-172081-big-Data.db 1.3 GB 56.55% 0
18 la-172206-big-Data.db 125.7 MB 54.66% 0
19
20
21Total size: 277.9 GB (data files only)

Eevans updated the task description. (Show Details)Nov 7 2016, 5:18 PM
Eevans added a comment.EditedNov 7 2016, 5:34 PM

Next round of conversions (local_group_*_title__revisions.{data,idx_by_rev_ever}).

1#!/bin/sh
2
3set -ex
4
5c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikisource_T_title__revisions\".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
6sleep 30
7c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikinews_T_title__revisions\".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
8sleep 30
9c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikiversity_T_title__revisions\".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
10sleep 30
11c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_phase0_T_title__revisions\".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
12sleep 30
13c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikibooks_T_title__revisions\".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
14sleep 30
15c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikiquote_T_title__revisions\".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
16sleep 30
17c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikivoyage_T_title__revisions\".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
18sleep 30
19c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wiktionary_T_title__revisions\".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
20sleep 30
21c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikimedia_T_title__revisions\".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
22sleep 30
23c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_default_T_title__revisions\".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
24sleep 30
25c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikipedia_T_title__revisions\".data WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
26sleep 30
27c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikisource_T_title__revisions\".idx_by_rev_ever WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
28sleep 30
29c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikinews_T_title__revisions\".idx_by_rev_ever WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
30sleep 30
31c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikiversity_T_title__revisions\".idx_by_rev_ever WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
32sleep 30
33c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_phase0_T_title__revisions\".idx_by_rev_ever WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
34sleep 30
35c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikibooks_T_title__revisions\".idx_by_rev_ever WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
36sleep 30
37c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikiquote_T_title__revisions\".idx_by_rev_ever WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
38sleep 30
39c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikivoyage_T_title__revisions\".idx_by_rev_ever WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
40sleep 30
41c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wiktionary_T_title__revisions\".idx_by_rev_ever WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
42sleep 30
43c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikimedia_T_title__revisions\".idx_by_rev_ever WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
44sleep 30
45c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_default_T_title__revisions\".idx_by_rev_ever WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
46sleep 30
47c-commands/c-cqlsh a -e "ALTER TABLE \"local_group_wikipedia_T_title__revisions\".idx_by_rev_ever WITH compaction = { 'compaction_window_unit': 'DAYS', 'compaction_window_size': '14', 'class':'com.jeffjirsa.cassandra.db.compaction.TimeWindowCompactionStrategy' };"
48sleep 30

Eevans updated the task description. (Show Details)Nov 7 2016, 5:59 PM

Mentioned in SAL (#wikimedia-operations) [2016-11-07T18:00:34Z] <urandom> T133395: Convert local_group_*_title__revisions.{data,idx_by_rev_ever} tables to time-window compaction

Eevans updated the task description. (Show Details)Nov 7 2016, 6:26 PM

Mentioned in SAL (#wikimedia-operations) [2016-11-07T20:44:29Z] <urandom> T133395: restbase2001-b.codfw.wmnet: Performing user-defined compaction of la-169239-big-Data.db and la-172629-big-Data.db

Mentioned in SAL (#wikimedia-operations) [2016-11-08T19:22:36Z] <urandom> T133395: Converting 25 additional RESTBase tables to TWCS

Eevans added a comment.Nov 8 2016, 7:51 PM

The results of time-window compaction testing have been positive, so (and as discussed in this morning's Services priortization meeting), I'm going to move forward with the wholesale conversion of all applicable keyspaces.

I will plan on converting the remaining 125 tables in 25 table batches, allowing sufficient time between batches for compaction to quiesce; I expect that we'll have all tables converted by no later than weeks end.

The first 25 tables:

local_group_default_T_feed_aggregated
local_group_default_T_feed_aggregated_historic
local_group_default_T_parsoid_dataDVIsgzJSne8kBh
local_group_default_T_parsoid_dataW4ULtxs1oMqJeY
local_group_default_T_parsoid_html
local_group_default_T_parsoid_section_offsets
local_group_default_T_parsoid_stash_datU81yvllO3
local_group_default_T_parsoid_stash_html
local_group_default_T_parsoid_stash_wikitext
local_group_default_T_parsoid_wikitext
local_group_default_T_summary
local_group_globaldomain_T_mathoid_check
local_group_globaldomain_T_mathoid_data
local_group_globaldomain_T_mathoid_hash__table
local_group_globaldomain_T_mathoid_input
local_group_globaldomain_T_mathoid_mml
local_group_globaldomain_T_mathoid_png
local_group_globaldomain_T_mathoid_request
local_group_globaldomain_T_mathoid_svg
local_group_phase0_T_feed_aggregated
local_group_phase0_T_feed_aggregated_historic
local_group_phase0_T_parsoid_dataDVIsgzJSne8kBhf
local_group_phase0_T_parsoid_html
local_group_phase0_T_parsoid_section_offsets
local_group_phase0_T_parsoid_dataW4ULtxs1oMqJeY1
Eevans added a comment.Nov 9 2016, 3:47 PM

The next 25:

local_group_phase0_T_parsoid_stash_dataU81yvllO3
local_group_phase0_T_parsoid_stash_html
local_group_phase0_T_parsoid_stash_wikitext
local_group_phase0_T_parsoid_wikitext
local_group_phase0_T_summary
local_group_wikibooks_T_feed_aggregated
local_group_wikibooks_T_feed_aggregated_historic
local_group_wikibooks_T_parsoid_dataDVIsgzJSne8k
local_group_wikibooks_T_parsoid_dataW4ULtxs1oMqJ
local_group_wikibooks_T_parsoid_html
local_group_wikibooks_T_parsoid_section_offsets
local_group_wikibooks_T_parsoid_stash_daU81yvllO
local_group_wikibooks_T_parsoid_stash_html
local_group_wikibooks_T_parsoid_stash_wikitext
local_group_wikibooks_T_parsoid_wikitext
local_group_wikibooks_T_summary
local_group_wikimedia_T_feed_aggregated
local_group_wikimedia_T_feed_aggregated_historic
local_group_wikimedia_T_parsoid_dataDVIsgzJSne8k
local_group_wikimedia_T_parsoid_dataW4ULtxs1oMqJ
local_group_wikimedia_T_parsoid_section_offsets
local_group_wikimedia_T_parsoid_stash_daU81yvllO
local_group_wikimedia_T_parsoid_stash_html
local_group_wikimedia_T_parsoid_stash_wikitext
local_group_wikimedia_T_parsoid_wikitext

Mentioned in SAL (#wikimedia-operations) [2016-11-09T15:47:21Z] <urandom> T133395: Converting the next 25 RESTBase keyspaces to TWCS

Eevans added a comment.Nov 9 2016, 5:22 PM

A couple more user-defined compaction experiments:

restbase2001-b, wikipedia_T_parsoid_html, prior to compaction:

1$ sudo ./inspect /srv/cassandra-b/data/local_group_wikipedia_T_parsoid_html/data-f3648bc0c2cb11e49ce6a1da77f2fd34/
2 File Size Tombstones Repaired at
32016-09-22
4 la-169239-big-Data.db 257.7 GB 0.07% 0
5
62016-10-06
7 la-172629-big-Data.db 22 GB 83.17% 0
8
92016-10-20
10 la-172681-big-Data.db 17.9 GB 83.17% 0
11
122016-11-03
13 la-172739-big-Data.db 4.1 GB 84.99% 0
14 la-172784-big-Data.db 29.6 MB 55.29% 0
15 la-172785-big-Data.db 41.5 MB 53.39% 0
16 la-172627-big-Data.db 1.9 GB 61.91% 0
17 la-172558-big-Data.db 1.2 GB 71.53% 0
18 la-172543-big-Data.db 3.2 GB 84.59% 0
19 la-172757-big-Data.db 410 MB 53.94% 0
20 la-172714-big-Data.db 1.7 GB 59.73% 0
21 la-172735-big-Data.db 335.8 MB 54.42% 0
22 la-172778-big-Data.db 438.7 MB 54.34% 0
23 la-172786-big-Data.db 38.4 MB 55.51% 0
24 la-172783-big-Data.db 124.4 MB 52.78% 0
25
26
27Total size: 311.1 GB (data files only)
28$

After compacting la-169239-big-Data.db and la-172629-big-Data.db (estimated droppable tombstones of 0.07% and 83.17% respectively):

1$ sudo ./inspect /srv/cassandra-b/data/local_group_wikipedia_T_parsoid_html/data-f3648bc0c2cb11e49ce6a1da77f2fd34/
2 File Size Tombstones Repaired at
32016-10-06
4 la-172789-big-Data.db 270.3 GB 0.51% 0
5
62016-10-20
7 la-172921-big-Data.db 17.9 GB 83.13% 0
8
92016-11-03
10 la-173138-big-Data.db 90.3 MB 50.18% 0
11 la-173133-big-Data.db 90.4 MB 55.95% 0
12 la-172947-big-Data.db 4.1 GB 84.90% 0
13 la-173086-big-Data.db 281.2 MB 56.39% 0
14 la-173065-big-Data.db 1.5 GB 58.26% 0
15 la-173143-big-Data.db 66.8 MB 59.57% 0
16 la-172792-big-Data.db 3.2 GB 84.59% 0
17 la-172893-big-Data.db 1.5 GB 81.73% 0
18 la-173128-big-Data.db 416.2 MB 57.61% 0
19 la-172805-big-Data.db 5.2 GB 81.12% 0
20 la-173107-big-Data.db 358.1 MB 54.68% 0
21 la-172980-big-Data.db 1.3 GB 75.23% 0
22
23
24Total size: 306.2 GB (data files only)

This results in a new table, la-172789-big-Data.db, with 0.51%.


And another: restbase2001-a, wiktionary_T_parsoid_html:

  File                                 Size       Tombstones     Repaired at
2015-08-27
  la-47424-big-Data.db               4.6 kB            0.00%   1477925317318

2015-10-22
  la-52590-big-Data.db               1.3 kB            0.00%   1477925317318

2016-01-28
  la-52588-big-Data.db                37 kB           50.56%   1477925317318

2016-02-25
  la-52586-big-Data.db                 1 MB           48.54%   1477925317318

2016-04-07
  la-52584-big-Data.db               1.8 MB           64.00%   1477925317318

2016-07-28
  la-52578-big-Data.db               1.6 GB           78.78%   1477925317318

2016-08-11
  la-52580-big-Data.db               1.6 GB           81.51%   1477925317318

2016-10-20
  la-88252-big-Data.db               8.1 GB           82.11%   1477925317318

2016-11-03
  la-88246-big-Data.db              58.2 MB           83.86%   1478554877199
  la-81778-big-Data.db              50.5 MB           79.64%   1478207818401
  la-88247-big-Data.db             148.3 kB           62.67%   1478554877199
  la-83683-big-Data.db              68.8 MB           84.38%   1478554877199
  la-88254-big-Data.db              52.6 MB           74.24%               0
  la-88242-big-Data.db             258.9 MB           83.39%   1478554877199
  la-88263-big-Data.db              24.1 MB           58.80%               0

Total size: 11.9 GB (data files only)

After compacting:

  • la-47424-big-Data.db
  • la-52590-big-Data.db
  • la-52588-big-Data.db
  • la-52586-big-Data.db
  • la-52584-big-Data.db
  File                                 Size       Tombstones     Repaired at
2016-07-28
  la-52578-big-Data.db               1.6 GB           78.78%   1477925317318

2016-08-11
  la-88264-big-Data.db               1.6 GB           81.49%   1477925317318

2016-10-20
  la-88252-big-Data.db               8.1 GB           82.11%   1477925317318

2016-11-03
  la-88246-big-Data.db              58.2 MB           83.89%   1478554877199
  la-81778-big-Data.db              50.5 MB           79.66%   1478207818401
  la-88247-big-Data.db             148.3 kB           62.82%   1478554877199
  la-83683-big-Data.db              68.8 MB           84.41%   1478554877199
  la-88254-big-Data.db              52.6 MB           74.54%               0
  la-88242-big-Data.db             258.9 MB           83.39%   1478554877199
  la-88263-big-Data.db              24.1 MB           59.04%               0


Total size: 11.9 GB (data files only)

And again, after a second pass to compact:

  • la-52578-big-Data.db
  • la-88264-big-Data.db
  • la-88252-big-Data.db
  File                                 Size       Tombstones     Repaired at
2016-10-20
  la-88265-big-Data.db               7.6 GB           29.96%   1477925317318

2016-11-03
  la-88246-big-Data.db              58.2 MB           84.04%   1478554877199
  la-88266-big-Data.db               5.9 MB           54.71%               0
  la-81778-big-Data.db              50.5 MB           79.70%   1478207818401
  la-88247-big-Data.db             148.3 kB           86.23%   1478554877199
  la-83683-big-Data.db              68.8 MB           84.49%   1478554877199
  la-88254-big-Data.db              52.6 MB           81.81%               0
  la-88242-big-Data.db             258.9 MB           83.39%   1478554877199
  la-88263-big-Data.db              24.1 MB           59.93%               0


Total size: 8.1 GB (data files only)
Eevans added a comment.Nov 9 2016, 8:30 PM

Next 25:

local_group_wikimedia_T_summary
local_group_wikinews_T_feed_aggregated
local_group_wikinews_T_feed_aggregated_historic
local_group_wikinews_T_parsoid_dataDVIsgzJSne8kB
local_group_wikinews_T_parsoid_dataW4ULtxs1oMqJe
local_group_wikinews_T_parsoid_html
local_group_wikinews_T_parsoid_section_offsets
local_group_wikinews_T_parsoid_stash_daU81yvllO3
local_group_wikinews_T_parsoid_stash_html
local_group_wikinews_T_parsoid_stash_wikitext
local_group_wikinews_T_parsoid_wikitext
local_group_wikinews_T_summary
local_group_wikipedia_T_feed_aggregated
local_group_wikipedia_T_feed_aggregated_historic
local_group_wikipedia_T_parsoid_dataDVIsgzJSne8k
local_group_wikipedia_T_parsoid_dataW4ULtxs1oMqJ
local_group_wikipedia_T_parsoid_section_offsets
local_group_wikipedia_T_parsoid_stash_daU81yvllO
local_group_wikipedia_T_parsoid_stash_html
local_group_wikipedia_T_parsoid_stash_wikitext
local_group_wikipedia_T_parsoid_wikitext
local_group_wikipedia_T_summary
local_group_wikiquote_T_feed_aggregated
local_group_wikiquote_T_feed_aggregated_historic
local_group_wikiquote_T_parsoid_dataDVIsgzJSne8k

Mentioned in SAL (#wikimedia-operations) [2016-11-09T20:32:01Z] <urandom> T133395: Converting next 25 RESTBase tables to time-window compaction

Eevans updated the task description. (Show Details)Nov 9 2016, 11:02 PM

Next 25 conversions:

local_group_wikiquote_T_parsoid_dataW4ULtxs1oMqJ
local_group_wikiquote_T_parsoid_html
local_group_wikiquote_T_parsoid_section_offsets
local_group_wikiquote_T_parsoid_stash_daU81yvllO
local_group_wikiquote_T_parsoid_stash_html
local_group_wikiquote_T_parsoid_stash_wikitext
local_group_wikiquote_T_parsoid_wikitext
local_group_wikiquote_T_summary
local_group_wikisource_T_feed_aggregated
local_group_wikisource_T_feed_aggregatedQGi0IBt
local_group_wikisource_T_parsoid_dataDVIsgzJSne8
local_group_wikisource_T_parsoid_dataW4ULtxs1oMq
local_group_wikisource_T_parsoid_html
local_group_wikisource_T_parsoid_section_offsets
local_group_wikisource_T_parsoid_stash_dU81yvllO
local_group_wikisource_T_parsoid_stash_html
local_group_wikisource_T_parsoid_stash_wikitext
local_group_wikisource_T_parsoid_wikitext
local_group_wikisource_T_summary
local_group_wikiversity_T_feed_aggregated
local_group_wikiversity_T_feed_aggregateQGi0IBt
local_group_wikiversity_T_parsoid_dataDVIsgzJSne
local_group_wikiversity_T_parsoid_dataW4ULtxs1oM
local_group_wikiversity_T_parsoid_html
local_group_wikiversity_T_parsoid_sectioeXH8_ri

Mentioned in SAL (#wikimedia-operations) [2016-11-10T16:58:54Z] <urandom> T133395: Performing next 25 RESTBase table conversions to TWCS

Final 25:

local_group_wikiversity_T_parsoid_stash_html
local_group_wikiversity_T_parsoid_stash_U81yvllO
local_group_wikiversity_T_parsoid_stash_wikitext
local_group_wikiversity_T_parsoid_wikitext
local_group_wikiversity_T_summary
local_group_wikivoyage_T_feed_aggregated
local_group_wikivoyage_T_feed_aggregatedQGi0IBt
local_group_wikivoyage_T_parsoid_dataDVIsgzJSne8
local_group_wikivoyage_T_parsoid_dataW4ULtxs1oMq
local_group_wikivoyage_T_parsoid_html
local_group_wikivoyage_T_parsoid_section_offsets
local_group_wikivoyage_T_parsoid_stash_dU81yvllO
local_group_wikivoyage_T_parsoid_stash_html
local_group_wikivoyage_T_parsoid_stash_wikitext
local_group_wikivoyage_T_parsoid_wikitext
local_group_wikivoyage_T_summary
local_group_wiktionary_T_parsoid_dataDVIsgzJSne8
local_group_wiktionary_T_parsoid_dataW4ULtxs1oMq
local_group_wiktionary_T_parsoid_section_offsets
local_group_wiktionary_T_parsoid_stash_dU81yvllO
local_group_wiktionary_T_parsoid_stash_html
local_group_wiktionary_T_parsoid_stash_wikitext
local_group_wiktionary_T_parsoid_wikitext
local_group_wiktionary_T_summary
local_group_wiktionary_T_term_definition

Mentioned in SAL (#wikimedia-operations) [2016-11-10T17:46:09Z] <urandom> T133395: Convert final 25 RESTBase tables to TWCS

Eevans renamed this task from Evaluate TimeWindowCompactionStrategy to Convert RESTBase Cassandra cluster to TimeWindowCompactionStrategy.Nov 29 2016, 9:41 PM
Eevans updated the task description. (Show Details)Nov 29 2016, 9:53 PM
Eevans moved this task from In-Progress to Blocked on the Cassandra board.
Eevans edited projects, added Services (blocked); removed Services (doing).
Eevans closed this task as Resolved.Feb 27 2017, 4:44 PM
Eevans claimed this task.

T159161: User-defined compaction work-around for tombstone GC has been opened to separately keep track of the user-defined compactions; The conversion to TWCS (the original scope of this issue), is complete.