Page MenuHomePhabricator
Create Task
Maniphest T180804

Reconfigure deflate compressed keyspaces to use LZ4
Closed, ResolvedPublic
Actions

Description

T179105 was created to change default compression from deflate, to LZ4 (with a chunk size of 64KB). Some production in keyspaces in the new cluster have already been created using deflate, and should be reconfigured.

Those keyspaces (at the time of this writing):

commons_T_feed_aggregated
commons_T_mobile__ng_lead
commons_T_mobile__ng_remaining
commons_T_page__restrictions
commons_T_page__summary
commons_T_parsoid__ng_data2pfX88WDfxDffCFHoTUDoy
commons_T_parsoid__ng_html
commons_T_parsoid__ng_section9wYKz1EvijuZtgPVP5t
enwiki_T_feed_aggregated
enwiki_T_mobile__ng_lead
enwiki_T_mobile__ng_remaining
enwiki_T_page__restrictions
enwiki_T_parsoid__ng_data2pfX88WDfxDffCFHoTUDoy8
enwiki_T_parsoid__ng_html
enwiki_T_parsoid__ng_section9wYKz1EvijuZtgPVP5tq
local_group_globaldomain_T_mathoid_mml
local_group_globaldomain_T_mathoid_svg
others_T_feed_aggregated
others_T_mobile__ng_lead
others_T_mobile__ng_remaining
others_T_page__restrictions
others_T_page__summary
others_T_parsoid__ng_data2pfX88WDfxDffCFHoTUDoy8
others_T_parsoid__ng_html
others_T_parsoid__ng_section9wYKz1EvijuZtgPVP5tq
wikipedia_T_feed_aggregated
wikipedia_T_mobile__ng_lead
wikipedia_T_mobile__ng_remaining
wikipedia_T_page__restrictions
wikipedia_T_page__summary
wikipedia_T_parsoid__ng_data2pfX88WDfxDffCFHoTUD
wikipedia_T_parsoid__ng_html
wikipedia_T_parsoid__ng_section9wYKz1EvijuZtgPVP

An [[ https://github.com/eevans/cassandra-schema-nodejs/blob/master/bin/mkschema | mkschema ]] compatible set of ALTERs:

P6343 compression.yaml
1commons_T_feed_aggregated:
2 statement: |
3 ALTER TABLE "commons_T_feed_aggregated".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
4commons_T_mobile__ng_lead:
5 statement: |
6 ALTER TABLE "commons_T_mobile__ng_lead".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
7commons_T_mobile__ng_remaining:
8 statement: |
9 ALTER TABLE "commons_T_mobile__ng_remaining".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
10commons_T_page__restrictions:
11 statement: |
12 ALTER TABLE "commons_T_page__restrictions".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
13commons_T_page__summary:
14 statement: |
15 ALTER TABLE "commons_T_page__summary".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
16commons_T_parsoid__ng_data2pfX88WDfxDffCFHoTUDoy:
17 statement: |
18 ALTER TABLE "commons_T_parsoid__ng_data2pfX88WDfxDffCFHoTUDoy".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
19commons_T_parsoid__ng_html:
20 statement: |
21 ALTER TABLE "commons_T_parsoid__ng_html".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
22commons_T_parsoid__ng_section9wYKz1EvijuZtgPVP5t:
23 statement: |
24 ALTER TABLE "commons_T_parsoid__ng_section9wYKz1EvijuZtgPVP5t".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
25enwiki_T_feed_aggregated:
26 statement: |
27 ALTER TABLE "enwiki_T_feed_aggregated".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
28enwiki_T_mobile__ng_lead:
29 statement: |
30 ALTER TABLE "enwiki_T_mobile__ng_lead".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
31enwiki_T_mobile__ng_remaining:
32 statement: |
33 ALTER TABLE "enwiki_T_mobile__ng_remaining".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
34enwiki_T_page__restrictions:
35 statement: |
36 ALTER TABLE "enwiki_T_page__restrictions".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
37enwiki_T_parsoid__ng_data2pfX88WDfxDffCFHoTUDoy8:
38 statement: |
39 ALTER TABLE "enwiki_T_parsoid__ng_data2pfX88WDfxDffCFHoTUDoy8".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
40enwiki_T_parsoid__ng_html:
41 statement: |
42 ALTER TABLE "enwiki_T_parsoid__ng_html".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
43enwiki_T_parsoid__ng_section9wYKz1EvijuZtgPVP5tq:
44 statement: |
45 ALTER TABLE "enwiki_T_parsoid__ng_section9wYKz1EvijuZtgPVP5tq".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
46local_group_globaldomain_T_mathoid_mml:
47 statement: |
48 ALTER TABLE "local_group_globaldomain_T_mathoid_mml".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
49local_group_globaldomain_T_mathoid_svg:
50 statement: |
51 ALTER TABLE "local_group_globaldomain_T_mathoid_svg".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
52others_T_feed_aggregated:
53 statement: |
54 ALTER TABLE "others_T_feed_aggregated".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
55others_T_mobile__ng_lead:
56 statement: |
57 ALTER TABLE "others_T_mobile__ng_lead".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
58others_T_mobile__ng_remaining:
59 statement: |
60 ALTER TABLE "others_T_mobile__ng_remaining".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
61others_T_page__restrictions:
62 statement: |
63 ALTER TABLE "others_T_page__restrictions".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
64others_T_page__summary:
65 statement: |
66 ALTER TABLE "others_T_page__summary".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
67others_T_parsoid__ng_data2pfX88WDfxDffCFHoTUDoy8:
68 statement: |
69 ALTER TABLE "others_T_parsoid__ng_data2pfX88WDfxDffCFHoTUDoy8".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
70others_T_parsoid__ng_html:
71 statement: |
72 ALTER TABLE "others_T_parsoid__ng_html".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
73others_T_parsoid__ng_section9wYKz1EvijuZtgPVP5tq:
74 statement: |
75 ALTER TABLE "others_T_parsoid__ng_section9wYKz1EvijuZtgPVP5tq".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
76wikipedia_T_feed_aggregated:
77 statement: |
78 ALTER TABLE "wikipedia_T_feed_aggregated".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
79wikipedia_T_mobile__ng_lead:
80 statement: |
81 ALTER TABLE "wikipedia_T_mobile__ng_lead".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
82wikipedia_T_mobile__ng_remaining:
83 statement: |
84 ALTER TABLE "wikipedia_T_mobile__ng_remaining".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
85wikipedia_T_page__restrictions:
86 statement: |
87 ALTER TABLE "wikipedia_T_page__restrictions".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
88wikipedia_T_page__summary:
89 statement: |
90 ALTER TABLE "wikipedia_T_page__summary".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
91wikipedia_T_parsoid__ng_data2pfX88WDfxDffCFHoTUD:
92 statement: |
93 ALTER TABLE "wikipedia_T_parsoid__ng_data2pfX88WDfxDffCFHoTUD".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
94wikipedia_T_parsoid__ng_html:
95 statement: |
96 ALTER TABLE "wikipedia_T_parsoid__ng_html".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
97wikipedia_T_parsoid__ng_section9wYKz1EvijuZtgPVP:
98 statement: |
99 ALTER TABLE "wikipedia_T_parsoid__ng_section9wYKz1EvijuZtgPVP".data WITH {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}

NOTE: This should be done outside any bootstraps happening as a result of T179422: Reshape RESTBase Cassandra clusters

Related Objects
Search...

Event Timeline

Eevans created this task.Nov 17 2017, 4:16 PM
Eevans updated the task description. (Show Details)
Eevans moved this task from Backlog to Next on the User-Eevans board.
Eevans moved this task from Next to Blocked on the User-Eevans board.Nov 17 2017, 5:01 PM
Stashbot added a comment.Nov 20 2017, 3:59 PM

Mentioned in SAL (#wikimedia-operations) [2017-11-20T15:59:35Z] <urandom> Use lz4 compression instead of deflate (T180804)

Eevans closed this task as Resolved.Nov 20 2017, 5:11 PM

This is complete.

NOTE: If any more tables are completed before T179105 is completed, they will need to be manually changed as well.
fgiunchedi awarded a token.Nov 21 2017, 10:45 AM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL