Page MenuHomePhabricator

Create new mobile storage tables
Closed, ResolvedPublic

Description

We need to create keyspaces for the new mobile storage, where the data table should look somewhat like this:

CREATE TABLE "wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".data (
    "_domain" text,
    key text,
    headers text,
    tid timeuuid,
    value blob,
    PRIMARY KEY (("_domain", key))
) WITH bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
    AND comment = ''
    AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
    AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND crc_check_chance = 1.0
    AND dclocal_read_repair_chance = 0.1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99PERCENTILE';

With appropriate compaction/compression/replication settings.

In staging/production KS names would be:

"others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"
"wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"
"enwiki_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"

In the beta cluster the names would be:

"others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"

Event Timeline

Pchelolo created this task.Apr 15 2019, 7:45 PM

Production/staging DDL here

1create_others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
2 statement: |
3 CREATE KEYSPACE IF NOT EXISTS "others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"
4 WITH replication = {'class': 'NetworkTopologyStrategy', 'codfw': '3', 'eqiad': '3'} AND durable_writes = true
5
6create_meta_others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
7 statement: |
8 CREATE TABLE IF NOT EXISTS "others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".meta (
9 key text PRIMARY KEY,
10 value text
11 ) WITH bloom_filter_fp_chance = 0.1
12 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
13 AND comment = ''
14 AND compaction = {'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
15 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
16 AND crc_check_chance = 1.0
17 AND dclocal_read_repair_chance = 0.1
18 AND default_time_to_live = 0
19 AND gc_grace_seconds = 864000
20 AND max_index_interval = 2048
21 AND memtable_flush_period_in_ms = 0
22 AND min_index_interval = 128
23 AND read_repair_chance = 0.0
24 AND speculative_retry = '99PERCENTILE';
25
26create_data_others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
27 statement: |
28 CREATE TABLE IF NOT EXISTS "others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".data (
29 "_domain" text,
30 key text,
31 headers text,
32 tid timeuuid,
33 value blob,
34 PRIMARY KEY (("_domain", key))
35 ) WITH bloom_filter_fp_chance = 0.01
36 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
37 AND comment = ''
38 AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
39 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
40 AND crc_check_chance = 1.0
41 AND dclocal_read_repair_chance = 0.1
42 AND default_time_to_live = 0
43 AND gc_grace_seconds = 864000
44 AND max_index_interval = 2048
45 AND memtable_flush_period_in_ms = 0
46 AND min_index_interval = 128
47 AND read_repair_chance = 0.0
48 AND speculative_retry = '99PERCENTILE';
49
50create_wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
51 statement: |
52 CREATE KEYSPACE IF NOT EXISTS "wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"
53 WITH replication = {'class': 'NetworkTopologyStrategy', 'codfw': '3', 'eqiad': '3'} AND durable_writes = true
54
55create_meta_wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
56 statement: |
57 CREATE TABLE IF NOT EXISTS "wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".meta (
58 key text PRIMARY KEY,
59 value text
60 ) WITH bloom_filter_fp_chance = 0.1
61 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
62 AND comment = ''
63 AND compaction = {'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
64 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
65 AND crc_check_chance = 1.0
66 AND dclocal_read_repair_chance = 0.1
67 AND default_time_to_live = 0
68 AND gc_grace_seconds = 864000
69 AND max_index_interval = 2048
70 AND memtable_flush_period_in_ms = 0
71 AND min_index_interval = 128
72 AND read_repair_chance = 0.0
73 AND speculative_retry = '99PERCENTILE';
74
75create_data_wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
76 statement: |
77 CREATE TABLE IF NOT EXISTS "wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".data (
78 "_domain" text,
79 key text,
80 headers text,
81 tid timeuuid,
82 value blob,
83 PRIMARY KEY (("_domain", key))
84 ) WITH bloom_filter_fp_chance = 0.01
85 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
86 AND comment = ''
87 AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
88 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
89 AND crc_check_chance = 1.0
90 AND dclocal_read_repair_chance = 0.1
91 AND default_time_to_live = 0
92 AND gc_grace_seconds = 864000
93 AND max_index_interval = 2048
94 AND memtable_flush_period_in_ms = 0
95 AND min_index_interval = 128
96 AND read_repair_chance = 0.0
97 AND speculative_retry = '99PERCENTILE';
98
99create_enwiki_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
100 statement: |
101 CREATE KEYSPACE IF NOT EXISTS "enwiki_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"
102 WITH replication = {'class': 'NetworkTopologyStrategy', 'codfw': '3', 'eqiad': '3'} AND durable_writes = true
103
104create_meta_enwiki_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
105 statement: |
106 CREATE TABLE IF NOT EXISTS "enwiki_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".meta (
107 key text PRIMARY KEY,
108 value text
109 ) WITH bloom_filter_fp_chance = 0.1
110 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
111 AND comment = ''
112 AND compaction = {'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
113 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
114 AND crc_check_chance = 1.0
115 AND dclocal_read_repair_chance = 0.1
116 AND default_time_to_live = 0
117 AND gc_grace_seconds = 864000
118 AND max_index_interval = 2048
119 AND memtable_flush_period_in_ms = 0
120 AND min_index_interval = 128
121 AND read_repair_chance = 0.0
122 AND speculative_retry = '99PERCENTILE';
123
124create_data_enwiki_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
125 statement: |
126 CREATE TABLE IF NOT EXISTS "enwiki_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".data (
127 "_domain" text,
128 key text,
129 headers text,
130 tid timeuuid,
131 value blob,
132 PRIMARY KEY (("_domain", key))
133 ) WITH bloom_filter_fp_chance = 0.01
134 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
135 AND comment = ''
136 AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
137 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
138 AND crc_check_chance = 1.0
139 AND dclocal_read_repair_chance = 0.1
140 AND default_time_to_live = 0
141 AND gc_grace_seconds = 864000
142 AND max_index_interval = 2048
143 AND memtable_flush_period_in_ms = 0
144 AND min_index_interval = 128
145 AND read_repair_chance = 0.0
146 AND speculative_retry = '99PERCENTILE';
147

Deployment-prep to follow...

Deployment-prep:

1create_others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
2 statement: |
3 CREATE KEYSPACE IF NOT EXISTS "others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"
4 WITH replication = {'class': 'NetworkTopologyStrategy', 'codfw': '3', 'eqiad': '3'} AND durable_writes = true
5
6create_meta_others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
7 statement: |
8 CREATE TABLE IF NOT EXISTS "others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".meta (
9 key text PRIMARY KEY,
10 value text
11 ) WITH bloom_filter_fp_chance = 0.1
12 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
13 AND comment = ''
14 AND compaction = {'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
15 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
16 AND crc_check_chance = 1.0
17 AND dclocal_read_repair_chance = 0.1
18 AND default_time_to_live = 0
19 AND gc_grace_seconds = 864000
20 AND max_index_interval = 2048
21 AND memtable_flush_period_in_ms = 0
22 AND min_index_interval = 128
23 AND read_repair_chance = 0.0
24 AND speculative_retry = '99PERCENTILE';
25
26create_data_others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
27 statement: |
28 CREATE TABLE IF NOT EXISTS "others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".data (
29 "_domain" text,
30 key text,
31 headers text,
32 tid timeuuid,
33 value blob,
34 PRIMARY KEY (("_domain", key))
35 ) WITH bloom_filter_fp_chance = 0.01
36 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
37 AND comment = ''
38 AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
39 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
40 AND crc_check_chance = 1.0
41 AND dclocal_read_repair_chance = 0.1
42 AND default_time_to_live = 0
43 AND gc_grace_seconds = 864000
44 AND max_index_interval = 2048
45 AND memtable_flush_period_in_ms = 0
46 AND min_index_interval = 128
47 AND read_repair_chance = 0.0
48 AND speculative_retry = '99PERCENTILE';

LGTM!

Let's first execute in deployment-prep and follow into production after we merge/deploy https://github.com/wikimedia/restbase/pull/1117 in beta to verify all good and correct?

+1, sounds like a plan!

LGTM!

Let's first execute in deployment-prep and follow into production after we merge/deploy https://github.com/wikimedia/restbase/pull/1117 in beta to verify all good and correct?

OK; Applied to deployment-prep

Mentioned in SAL (#wikimedia-operations) [2019-04-16T00:33:28Z] <urandom> creating new restbase schema -- T221031

This has been applied to the production cluster

Eevans closed this task as Resolved.Apr 16 2019, 12:52 AM

Applied to the dev cluster; Done