Optimize logging table
Closed, ResolvedPublic

Description

After T184485 and T189596, there were a lot of deleted rows from the logging table on many wikis. This is a summary of the numbers of rows deleted:

wikidatawiki654029578
commonswiki196956373
frwiki84965885
itwiki43086794
enwiktionary40325873
shwiki38929189
dewiki30377762
viwiki29115179
frwiktionary17784536
nlwiki16476995

While the rows have been already physically deleted, optimizing the tables will reclaim (in some cases) a lot of space and clean up the internal purging log (in some cases, fixing slowdowns. eg. dbstore2* T189596#4160236 ).

While this is not high priority, it would be nice to keep track of the optimized databases/hosts to not work twice.

jcrespo created this task.Jun 15 2018, 10:39 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 15 2018, 10:39 AM
jcrespo triaged this task as Low priority.Jun 15 2018, 10:39 AM
jcrespo updated the task description. (Show Details)

wikidatawiki and commons were already done as there was a schema change that involved logging table and it rebuilt that table, and it was done right after the deletes finished on that table, so we mark those as done.

jcrespo updated the task description. (Show Details)Jun 15 2018, 10:46 AM
jcrespo updated the task description. (Show Details)Jun 15 2018, 10:51 AM
jcrespo moved this task from Triage to Meta/Epic on the DBA board.Jun 15 2018, 11:28 AM
Nirmos added a subscriber: Nirmos.Jun 16 2018, 11:56 AM
Vvjjkkii renamed this task from Optimize logging table to cuaaaaaaaa.Sun, Jul 1, 1:03 AM
Vvjjkkii raised the priority of this task from Low to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
Marostegui renamed this task from cuaaaaaaaa to Optimize logging table.Sun, Jul 1, 8:06 PM
Marostegui lowered the priority of this task from High to Low.
Marostegui updated the task description. (Show Details)
Marostegui updated the task description. (Show Details)

Mentioned in SAL (#wikimedia-operations) [2018-07-02T13:15:51Z] <marostegui> Optimize viwiki.logging on codfw - T197459

Marostegui claimed this task.EditedMon, Jul 2, 1:15 PM
Marostegui moved this task from Meta/Epic to In progress on the DBA board.

As I am altering s7, where frwiktionary and viwiki lives, I will take the opportunity to optimize logging table there

Mentioned in SAL (#wikimedia-operations) [2018-07-02T13:21:07Z] <marostegui> Optimize frwiktionary'.logging on codfw - T197459

@Marostegui Perhaps the description of this task needs an update after your action.

@Marostegui Perhaps the description of this task needs an update after your action.

No because it has not finished across all the hosts of S7 ;)

@Marostegui Perhaps the description of this task needs an update after your action.

No because it has not finished across all the hosts of S7 ;)

I'm glad I didn't make the change myself :)

Mentioned in SAL (#wikimedia-operations) [2018-07-03T04:40:21Z] <marostegui> Deploy schema change on db1086 T191316 T192926 T89737 T195193 T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-03T05:26:33Z] <marostegui> Deploy schema change on db1094 T191316 T192926 T89737 T195193 T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-03T06:14:24Z] <marostegui> Deploy schema change on db1079 with replication, this will generate lag on s7 on labsdb hosts T191316 T192926 T89737 T195193 T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-03T07:13:34Z] <marostegui> Deploy schema change on db1090:3317 T191316 T192926 T89737 T195193 T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-03T09:13:13Z] <marostegui> Deploy schema change on db1098:3317 T191316 T192926 T89737 T195193 T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-03T09:58:59Z] <marostegui> Deploy schema change on s3 codfw primary master (db2043) with replication, this will generate lag on s3 codfw T191316 T192926 T89737 T195193 T197459

Marostegui updated the task description. (Show Details)Tue, Jul 3, 1:57 PM

Mentioned in SAL (#wikimedia-operations) [2018-07-04T15:55:40Z] <marostegui> Optimize dewiki.logging on s5 codfw master with replication, this will generate lag on s5 codfw - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-05T06:09:48Z] <marostegui> Optimize dewiki.logging on db1100 and dbstore1002:s5 - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-05T07:40:12Z] <marostegui> Optimize dewiki.logging on db1096:3315 - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-05T09:50:48Z] <marostegui> Optimize dewiki.logging on db1097:3315 - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-05T13:18:43Z] <marostegui> Optimize dewiki.logging on db1113:3315 - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-05T15:00:04Z] <marostegui> Optimize dewiki.logging on db1082 with replication, this will generate lag on s5 on labsdb hosts/script load irssinotifier - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-06T05:01:49Z] <marostegui> Optimize dewiki.logging on db1110 - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-06T06:22:16Z] <marostegui> Optimize dewiki.logging on db1070 (s5 primary master) - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-06T06:31:25Z] <marostegui> Optimize frwiki.logging on db2039 (s6 codfw master), with replication, this will generate lag on s6 codfw - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-06T06:49:51Z] <marostegui> Optimize frwiki.logging on dbstore1002 - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-06T07:11:48Z] <marostegui> Optimize frwiki.logging on db1096:3316 - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-06T07:47:31Z] <marostegui> Optimize frwiki.logging on db1098:3316 - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-06T08:18:21Z] <marostegui> Optimize frwiki.logging on db1113:3316 - T197459

Marostegui updated the task description. (Show Details)Fri, Jul 6, 8:49 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T08:53:36Z] <marostegui> Optimize frwiki.logging on db11085 with replication (this will generate lag on s6 on labsdb hosts) - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-06T10:02:46Z] <marostegui> Optimize frwiki.logging on db1093 - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-06T10:49:33Z] <marostegui> Optimize frwiki.logging on db1088 - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-06T11:04:36Z] <marostegui> Optimize frwiki.logging on db1061 (s6 primary master) - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-06T11:08:25Z] <marostegui> Optimize shwiki.logging on db2043 (codfw s3 master), this will generate lag on s3 codfw - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-06T11:09:49Z] <marostegui> Optimize shwiki.logging on s3 eqiad hosts (one by one) T197459

Marostegui updated the task description. (Show Details)Fri, Jul 6, 11:12 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T11:15:34Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on db2035 (codfw s2 master), this will generate lag on s2 codfw - T197459

Marostegui updated the task description. (Show Details)Fri, Jul 6, 11:26 AM

Mentioned in SAL (#wikimedia-operations) [2018-07-06T11:51:20Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on dbstore1002:s2 - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-06T12:38:47Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on db1105 and db1122 - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-06T13:22:40Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on db1103:3312 - T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-09T05:04:18Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on db1074 with replication, this will generate lag on labs hosts T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-09T05:31:30Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on db1090 T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-09T06:13:44Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on db1076 T197459

Mentioned in SAL (#wikimedia-operations) [2018-07-09T06:41:19Z] <marostegui> Optimize {itwiki,enwiktionary,nlwiki}.logging on db1066 (s2 primary master) T197459

Marostegui updated the task description. (Show Details)Mon, Jul 9, 7:05 AM
Marostegui closed this task as Resolved.

This is all done