Page MenuHomePhabricator
Create Task
Maniphest T196024

Convert timeline keyspaces (indices) to time-windowed compaction
Closed, ResolvedPublic
Actions

Description

The indices used in k-r-v table revision retention are still using the default time-series compaction strategy, date-tiered (inefficient, and deprecated upstream). These should be converted to use time-windowed instead.

Keyspaces to convert:

keyspacetable name
commons_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLdsmobile_ng-revision-timeline
commons_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_wmobile_ng-render-timeline
commons_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04parsoid_ng-render-timeline
commons_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0parsoid_ng-revision-timeline
enwiki_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLdsmobile_ng-revision-timeline
enwiki_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_wmobile_ng-render-timeline
enwiki_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04parsoid_ng-render-timeline
enwiki_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0parsoid_ng-revision-timeline
others_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLdsmobile_ng-revision-timeline
others_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_wmobile_ng-render-timeline
others_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04parsoid_ng-render-timeline
others_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0parsoid_ng-revision-timeline
wikipedia_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLdmobile_ng-revision-timeline
wikipedia_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_mobile_ng-render-timeline
wikipedia_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVPparsoid_ng-render-timeline
wikipedia_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSparsoid_ng-revision-timeline

Related Objects

Event Timeline

Eevans created this task.May 30 2018, 9:32 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 30 2018, 9:32 PM
Eevans triaged this task as Medium priority.May 30 2018, 9:32 PM
Eevans updated the task description. (Show Details)May 30 2018, 9:57 PM
Eevans added a comment.May 30 2018, 9:59 PM

I propose the following, that will result in 2 week roll-ups.

P7193 timeline_alters.yaml
1alter_commons_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLds:
2 statement: |
3 ALTER TABLE "commons_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLds".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
4
5alter_commons_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_w:
6 statement: |
7 ALTER TABLE "commons_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_w".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
8
9alter_commons_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04:
10 statement: |
11 ALTER TABLE "commons_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
12
13alter_commons_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0:
14 statement: |
15 ALTER TABLE "commons_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
16
17alter_enwiki_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLds:
18 statement: |
19 ALTER TABLE "enwiki_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLds".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
20
21alter_enwiki_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_w:
22 statement: |
23 ALTER TABLE "enwiki_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_w".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
24
25alter_enwiki_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04:
26 statement: |
27 ALTER TABLE "enwiki_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
28
29alter_enwiki_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0:
30 statement: |
31 ALTER TABLE "enwiki_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
32
33alter_others_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLds:
34 statement: |
35 ALTER TABLE "others_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLds".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
36
37alter_others_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_w:
38 statement: |
39 ALTER TABLE "others_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_w".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
40
41alter_others_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04:
42 statement: |
43 ALTER TABLE "others_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP04".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
44
45alter_others_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0:
46 statement: |
47 ALTER TABLE "others_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWSp0".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
48
49alter_wikipedia_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLd:
50 statement: |
51 ALTER TABLE "wikipedia_T_mobile__ng3HeqOXXmkYfPizz4RPUR4OLXLd".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
52
53alter_wikipedia_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_:
54 statement: |
55 ALTER TABLE "wikipedia_T_mobile__ngR6XB1sh6_FFo_mfX4oZA56vpD_".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
56
57alter_wikipedia_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP:
58 statement: |
59 ALTER TABLE "wikipedia_T_parsoid__ng_4iO7iu2BPZmvx5_7kxNNNqVP".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
60
61alter_wikipedia_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWS:
62 statement: |
63 ALTER TABLE "wikipedia_T_parsoid__ngwEOh2HlqL6lr8QbNVKv1eSoWS".data WITH compaction = {'class': 'TimeWindowCompactionStrategy', 'compaction_window_size': '14', 'compaction_window_unit': 'DAYS'};
64

• mobrovac edited projects, added: Services (doing); removed: Services.May 31 2018, 12:19 PM
• mobrovac subscribed.

+1, LGTM.

Eevans moved this task from Backlog to In-Progress on the User-Eevans board.Jun 11 2018, 3:26 PM
Stashbot subscribed.Jun 11 2018, 6:15 PM

Mentioned in SAL (#wikimedia-operations) [2018-06-11T18:15:15Z] <urandom> convert timeline indices to time-windowed compaction - T196024

Eevans closed this task as Resolved.Jun 11 2018, 6:46 PM
• mobrovac edited projects, added: Services (done); removed: Services (doing).Jun 13 2018, 5:17 PM
• Vvjjkkii renamed this task from Convert timeline keyspaces (indices) to time-windowed compaction to 7xbaaaaaaa.Jul 1 2018, 1:07 AM
• Vvjjkkii reopened this task as Open.
• Vvjjkkii removed Eevans as the assignee of this task.
• Vvjjkkii raised the priority of this task from Medium to High.
• Vvjjkkii added projects: CheckUser, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), Tamil-Sites, Gamepress, Hashtags, Jade, KartoEditor, Language-2018-Apr-June, New-Editor-Experiences, Mail, TCB-Team (now WMDE-TechWish).
• Vvjjkkii updated the task description. (Show Details)
• Vvjjkkii removed a subscriber: Aklapper.
• mobrovac renamed this task from 7xbaaaaaaa to Convert timeline keyspaces (indices) to time-windowed compaction.Jul 1 2018, 11:10 AM
• mobrovac closed this task as Resolved.
• mobrovac assigned this task to Eevans.
• mobrovac lowered the priority of this task from High to Medium.
• mobrovac removed projects: TCB-Team (now WMDE-TechWish), Mail, New-Editor-Experiences, Language-2018-Apr-June, KartoEditor, Jade, Hashtags, Gamepress, Tamil-Sites, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), CheckUser.
• mobrovac updated the task description. (Show Details)
• mobrovac removed a subscriber: Eevans.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits