Page MenuHomePhabricator
Create Task
Maniphest T221031

Create new mobile storage tables
Closed, ResolvedPublic
Actions

Description

We need to create keyspaces for the new mobile storage, where the data table should look somewhat like this:

CREATE TABLE "wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".data (
    "_domain" text,
    key text,
    headers text,
    tid timeuuid,
    value blob,
    PRIMARY KEY (("_domain", key))
) WITH bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
    AND comment = ''
    AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
    AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND crc_check_chance = 1.0
    AND dclocal_read_repair_chance = 0.1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99PERCENTILE';

With appropriate compaction/compression/replication settings.

In staging/production KS names would be:

"others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"
"wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"
"enwiki_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"

In the beta cluster the names would be:

"others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"

Related Objects
Search...

Event Timeline

• Pchelolo created this task.Apr 15 2019, 7:45 PM
• mobrovac triaged this task as High priority.Apr 15 2019, 8:32 PM
• mobrovac edited projects, added Services (doing), Platform Team Workboards (Doing); removed Platform Team Legacy (Designing), Services (designing).
Eevans added a comment.Apr 15 2019, 8:36 PM

Production/staging DDL here

P8400 Masterwork From Distant Lands
1create_others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
2 statement: |
3 CREATE KEYSPACE IF NOT EXISTS "others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"
4 WITH replication = {'class': 'NetworkTopologyStrategy', 'codfw': '3', 'eqiad': '3'} AND durable_writes = true
5
6create_meta_others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
7 statement: |
8 CREATE TABLE IF NOT EXISTS "others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".meta (
9 key text PRIMARY KEY,
10 value text
11 ) WITH bloom_filter_fp_chance = 0.1
12 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
13 AND comment = ''
14 AND compaction = {'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
15 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
16 AND crc_check_chance = 1.0
17 AND dclocal_read_repair_chance = 0.1
18 AND default_time_to_live = 0
19 AND gc_grace_seconds = 864000
20 AND max_index_interval = 2048
21 AND memtable_flush_period_in_ms = 0
22 AND min_index_interval = 128
23 AND read_repair_chance = 0.0
24 AND speculative_retry = '99PERCENTILE';
25
26create_data_others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
27 statement: |
28 CREATE TABLE IF NOT EXISTS "others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".data (
29 "_domain" text,
30 key text,
31 headers text,
32 tid timeuuid,
33 value blob,
34 PRIMARY KEY (("_domain", key))
35 ) WITH bloom_filter_fp_chance = 0.01
36 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
37 AND comment = ''
38 AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
39 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
40 AND crc_check_chance = 1.0
41 AND dclocal_read_repair_chance = 0.1
42 AND default_time_to_live = 0
43 AND gc_grace_seconds = 864000
44 AND max_index_interval = 2048
45 AND memtable_flush_period_in_ms = 0
46 AND min_index_interval = 128
47 AND read_repair_chance = 0.0
48 AND speculative_retry = '99PERCENTILE';
49
50create_wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
51 statement: |
52 CREATE KEYSPACE IF NOT EXISTS "wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"
53 WITH replication = {'class': 'NetworkTopologyStrategy', 'codfw': '3', 'eqiad': '3'} AND durable_writes = true
54
55create_meta_wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
56 statement: |
57 CREATE TABLE IF NOT EXISTS "wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".meta (
58 key text PRIMARY KEY,
59 value text
60 ) WITH bloom_filter_fp_chance = 0.1
61 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
62 AND comment = ''
63 AND compaction = {'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
64 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
65 AND crc_check_chance = 1.0
66 AND dclocal_read_repair_chance = 0.1
67 AND default_time_to_live = 0
68 AND gc_grace_seconds = 864000
69 AND max_index_interval = 2048
70 AND memtable_flush_period_in_ms = 0
71 AND min_index_interval = 128
72 AND read_repair_chance = 0.0
73 AND speculative_retry = '99PERCENTILE';
74
75create_data_wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
76 statement: |
77 CREATE TABLE IF NOT EXISTS "wikipedia_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".data (
78 "_domain" text,
79 key text,
80 headers text,
81 tid timeuuid,
82 value blob,
83 PRIMARY KEY (("_domain", key))
84 ) WITH bloom_filter_fp_chance = 0.01
85 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
86 AND comment = ''
87 AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
88 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
89 AND crc_check_chance = 1.0
90 AND dclocal_read_repair_chance = 0.1
91 AND default_time_to_live = 0
92 AND gc_grace_seconds = 864000
93 AND max_index_interval = 2048
94 AND memtable_flush_period_in_ms = 0
95 AND min_index_interval = 128
96 AND read_repair_chance = 0.0
97 AND speculative_retry = '99PERCENTILE';
98
99create_enwiki_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
100 statement: |
101 CREATE KEYSPACE IF NOT EXISTS "enwiki_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"
102 WITH replication = {'class': 'NetworkTopologyStrategy', 'codfw': '3', 'eqiad': '3'} AND durable_writes = true
103
104create_meta_enwiki_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
105 statement: |
106 CREATE TABLE IF NOT EXISTS "enwiki_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".meta (
107 key text PRIMARY KEY,
108 value text
109 ) WITH bloom_filter_fp_chance = 0.1
110 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
111 AND comment = ''
112 AND compaction = {'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
113 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
114 AND crc_check_chance = 1.0
115 AND dclocal_read_repair_chance = 0.1
116 AND default_time_to_live = 0
117 AND gc_grace_seconds = 864000
118 AND max_index_interval = 2048
119 AND memtable_flush_period_in_ms = 0
120 AND min_index_interval = 128
121 AND read_repair_chance = 0.0
122 AND speculative_retry = '99PERCENTILE';
123
124create_data_enwiki_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
125 statement: |
126 CREATE TABLE IF NOT EXISTS "enwiki_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".data (
127 "_domain" text,
128 key text,
129 headers text,
130 tid timeuuid,
131 value blob,
132 PRIMARY KEY (("_domain", key))
133 ) WITH bloom_filter_fp_chance = 0.01
134 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
135 AND comment = ''
136 AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
137 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
138 AND crc_check_chance = 1.0
139 AND dclocal_read_repair_chance = 0.1
140 AND default_time_to_live = 0
141 AND gc_grace_seconds = 864000
142 AND max_index_interval = 2048
143 AND memtable_flush_period_in_ms = 0
144 AND min_index_interval = 128
145 AND read_repair_chance = 0.0
146 AND speculative_retry = '99PERCENTILE';
147

Deployment-prep to follow...

Eevans added a comment.Apr 15 2019, 8:39 PM

Deployment-prep:

P8401 Masterwork From Distant Lands
1create_others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
2 statement: |
3 CREATE KEYSPACE IF NOT EXISTS "others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY"
4 WITH replication = {'class': 'NetworkTopologyStrategy', 'codfw': '3', 'eqiad': '3'} AND durable_writes = true
5
6create_meta_others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
7 statement: |
8 CREATE TABLE IF NOT EXISTS "others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".meta (
9 key text PRIMARY KEY,
10 value text
11 ) WITH bloom_filter_fp_chance = 0.1
12 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
13 AND comment = ''
14 AND compaction = {'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
15 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
16 AND crc_check_chance = 1.0
17 AND dclocal_read_repair_chance = 0.1
18 AND default_time_to_live = 0
19 AND gc_grace_seconds = 864000
20 AND max_index_interval = 2048
21 AND memtable_flush_period_in_ms = 0
22 AND min_index_interval = 128
23 AND read_repair_chance = 0.0
24 AND speculative_retry = '99PERCENTILE';
25
26create_data_others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY:
27 statement: |
28 CREATE TABLE IF NOT EXISTS "others_T_mobileoZCBVtILw5eSrwi0VIGaFVSr2jY".data (
29 "_domain" text,
30 key text,
31 headers text,
32 tid timeuuid,
33 value blob,
34 PRIMARY KEY (("_domain", key))
35 ) WITH bloom_filter_fp_chance = 0.01
36 AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
37 AND comment = ''
38 AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
39 AND compression = {'chunk_length_in_kb': '32', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
40 AND crc_check_chance = 1.0
41 AND dclocal_read_repair_chance = 0.1
42 AND default_time_to_live = 0
43 AND gc_grace_seconds = 864000
44 AND max_index_interval = 2048
45 AND memtable_flush_period_in_ms = 0
46 AND min_index_interval = 128
47 AND read_repair_chance = 0.0
48 AND speculative_retry = '99PERCENTILE';

• Pchelolo added a comment.Apr 15 2019, 8:40 PM

LGTM!

Let's first execute in deployment-prep and follow into production after we merge/deploy https://github.com/wikimedia/restbase/pull/1117 in beta to verify all good and correct?

• mobrovac subscribed.Apr 15 2019, 8:46 PM

+1, sounds like a plan!

Eevans added a comment.Apr 15 2019, 9:10 PM
In T221031#5113112, @Pchelolo wrote:

LGTM!

Let's first execute in deployment-prep and follow into production after we merge/deploy https://github.com/wikimedia/restbase/pull/1117 in beta to verify all good and correct?

OK; Applied to deployment-prep

Stashbot added a comment.Apr 16 2019, 12:33 AM

Mentioned in SAL (#wikimedia-operations) [2019-04-16T00:33:28Z] <urandom> creating new restbase schema -- T221031

Eevans added a comment.Apr 16 2019, 12:49 AM

This has been applied to the production cluster

Eevans closed this task as Resolved.Apr 16 2019, 12:52 AM

Applied to the dev cluster; Done

• mobrovac edited projects, added Services (done), Platform Team Workboards (Done with CPT); removed Platform Team Workboards (Doing), Services (doing).Apr 16 2019, 8:15 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits