Page MenuHomePhabricator

Convert timeline keyspaces (indices) to time-windowed compaction
Closed, ResolvedPublic

Description

The indices used in k-r-v table revision retention are still using the default time-series compaction strategy, date-tiered (inefficient, and deprecated upstream). These should be converted to use time-windowed instead.

Keyspaces to convert:

keyspacetable name
commons_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLdsmobile_ng-revision-timeline
commons_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_wmobile_ng-render-timeline
commons_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04parsoid_ng-render-timeline
commons_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0parsoid_ng-revision-timeline
enwiki_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLdsmobile_ng-revision-timeline
enwiki_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_wmobile_ng-render-timeline
enwiki_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04parsoid_ng-render-timeline
enwiki_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0parsoid_ng-revision-timeline
others_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLdsmobile_ng-revision-timeline
others_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_wmobile_ng-render-timeline
others_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04parsoid_ng-render-timeline
others_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0parsoid_ng-revision-timeline
wikipedia_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLdmobile_ng-revision-timeline
wikipedia_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_mobile_ng-render-timeline
wikipedia_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVPparsoid_ng-render-timeline
wikipedia_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSparsoid_ng-revision-timeline

Related Objects

Event Timeline

Eevans created this task.May 30 2018, 9:32 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 30 2018, 9:32 PM
Eevans triaged this task as Medium priority.May 30 2018, 9:32 PM
Eevans updated the task description. (Show Details)May 30 2018, 9:57 PM

I propose the following, that will result in 2 week roll-ups.

1alter_commons_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLds:
2 statement: |
3 ALTER TABLE "commons_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLds".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
4
5alter_commons_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_w:
6 statement: |
7 ALTER TABLE "commons_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_w".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
8
9alter_commons_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04:
10 statement: |
11 ALTER TABLE "commons_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
12
13alter_commons_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0:
14 statement: |
15 ALTER TABLE "commons_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
16
17alter_enwiki_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLds:
18 statement: |
19 ALTER TABLE "enwiki_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLds".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
20
21alter_enwiki_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_w:
22 statement: |
23 ALTER TABLE "enwiki_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_w".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
24
25alter_enwiki_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04:
26 statement: |
27 ALTER TABLE "enwiki_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
28
29alter_enwiki_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0:
30 statement: |
31 ALTER TABLE "enwiki_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
32
33alter_others_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLds:
34 statement: |
35 ALTER TABLE "others_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLds".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
36
37alter_others_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_w:
38 statement: |
39 ALTER TABLE "others_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_w".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
40
41alter_others_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04:
42 statement: |
43 ALTER TABLE "others_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
44
45alter_others_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0:
46 statement: |
47 ALTER TABLE "others_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
48
49alter_wikipedia_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLd:
50 statement: |
51 ALTER TABLE "wikipedia_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLd".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
52
53alter_wikipedia_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_:
54 statement: |
55 ALTER TABLE "wikipedia_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
56
57alter_wikipedia_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP:
58 statement: |
59 ALTER TABLE "wikipedia_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
60
61alter_wikipedia_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWS:
62 statement: |
63 ALTER TABLE "wikipedia_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWS".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
64

mobrovac added a subscriber: mobrovac.

+1, LGTM.

Eevans moved this task from Backlog to In-Progress on the User-Eevans board.Jun 11 2018, 3:26 PM

Mentioned in SAL (#wikimedia-operations) [2018-06-11T18:15:15Z] <urandom> convert timeline indices to time-windowed compaction - T196024

Eevans closed this task as Resolved.Jun 11 2018, 6:46 PM
Vvjjkkii renamed this task from Convert timeline keyspaces (indices) to time-windowed compaction to 7xbaaaaaaa.Jul 1 2018, 1:07 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii removed Eevans as the assignee of this task.
Vvjjkkii raised the priority of this task from Medium to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
mobrovac renamed this task from 7xbaaaaaaa to Convert timeline keyspaces (indices) to time-windowed compaction.Jul 1 2018, 11:10 AM
mobrovac closed this task as Resolved.
mobrovac assigned this task to Eevans.
mobrovac lowered the priority of this task from High to Medium.
mobrovac updated the task description. (Show Details)
mobrovac removed a subscriber: Eevans.