@Ladsgroup can I assign this to you?

I prefer if someone else check what clean be cleaned up first. I can do the compression of the tables but cleaning up logs and such should be rather easy. Still, if noone does it by early next week, I can take over then.

Mentioned in SAL (#wikimedia-operations) [2023-06-06T05:34:00Z] <Amir1> ladsgroup@clouddb1021:/srv/sqldata.s1$ sudo rm db1196* (T337961)

The binlogs I deleted for s1 were not that old but it's okay. s3 binlogs were quite old though. deleted most of them.

I think templatelinks needs optimization in s3. I spot checked a wiki between different hosts (zhwiktionary) and templatelinks is quite large in clouddb1021. On it.

Hi @Ladsgroup - I'm planning on running a maintain-views on clouddb1021 at some point today as part of T315426.
It's going to recreate the views on one table on all databases on all shards.

Is this going to interfere with your tidy-up work at all, do you think? Would it be better for you if I were to defer the work?

Hi, The optimization of s3 is done so you can go ahead. I'll pick it up again tomorrow. It's much smaller that can wait for a while

Thanks, I've done that change now, so you can proceed with the tidy-up whenever you like.

Mentioned in SAL (#wikimedia-operations) [2023-06-13T18:27:13Z] <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb1021.eqiad.wmnet with reason: T337961

Mentioned in SAL (#wikimedia-operations) [2023-06-13T18:27:17Z] <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb1021.eqiad.wmnet with reason: T337961

Mentioned in SAL (#wikimedia-operations) [2023-06-13T18:29:11Z] <Amir1> root@clouddb1021.eqiad.wmnet[ruwikinews]> ALTER TABLE externallinks ROW_FORMAT=COMPRESSED; (T337961)

Mentioned in SAL (#wikimedia-operations) [2023-06-13T18:47:04Z] <Amir1> root@clouddb1021.eqiad.wmnet[commonswiki]> ALTER TABLE externallinks ROW_FORMAT=COMPRESSED; (T337961)

ruwikinews' externallinks:

218G externallinks.ibd

It needs to be compressed. I'll do a couple more tables too

This was a great breakthrough too: T338284#8928454

Yup, I think I'm going to compress templatelinks in commons too.

Mentioned in SAL (#wikimedia-operations) [2023-06-14T11:59:36Z] <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on clouddb1021.eqiad.wmnet with reason: T337961

Mentioned in SAL (#wikimedia-operations) [2023-06-14T11:59:49Z] <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on clouddb1021.eqiad.wmnet with reason: T337961

It's now at 79%, I compress another table and then call it done.

Mentioned in SAL (#wikimedia-operations) [2023-06-15T10:03:35Z] <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb1021.eqiad.wmnet with reason: T337961

Mentioned in SAL (#wikimedia-operations) [2023-06-15T10:03:39Z] <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb1021.eqiad.wmnet with reason: T337961

Mentioned in SAL (#wikimedia-operations) [2023-06-15T10:04:31Z] <Amir1> root@clouddb1021.eqiad.wmnet[metawiki]> ALTER TABLE pagelinks ROW_FORMAT=COMPRESSED; (T337961)