Page MenuHomePhabricator

Drop blob_tracking and blob_orphans everywhere
Closed, ResolvedPublic

Description

blob_tracking and blob_orphans are temporary tables which contain metadata relating to a single run of trackBlobs/recompressTracked. They need to be truncated each time the scripts are run. Currently they hold data related to a 2011 snapshot of the text table. Unfortunately we forgot to drop them after the last run of recompressTracked.php in 2011. They can be dropped now.

This does not mean that code related to these tables should be deleted. When we are ready to run the code again, we'll recreate the tables.


Original task description:

blob_tracking indexes apparently unused

On enwiki:

+---------------+------------+-----------+
| table_name    | index_name | rows_read |
+---------------+------------+-----------+
| blob_tracking | bt_moved   |      NULL |
| blob_tracking | bt_cluster |      NULL |
+---------------+------------+-----------+

NULL rows_read indicates the indexes havn't been used for reading or writing. Stats collection has been running here for over a week. Same results on S1 master and several slaves, and various other wikis.

Data / index disk usage for blob_tracking is roughly 50/50 split. Dropping unused indexes would be good to reduce write load and save disk space.

But are these indexes really likely to be simply unused? Is there any infrequent maintenance or reporting job that requires them?


Version: 1.21.x
Severity: minor

Dropping progress

  • s1
  • s2
  • s3
  • s4
  • s5
  • s6
  • s7
  • s8 (tables not present)

Event Timeline

bzimport raised the priority of this task from to Low.Nov 22 2014, 2:19 AM
bzimport set Reference to bz57186.
bzimport added a subscriber: Unknown Object (MLST).

(In reply to comment #0)

But are these indexes really likely to be simply unused? Is there any
infrequent maintenance or reporting job that requires them?

maintenance/storage/trackBlobs.php creates and populates this table.
maintenance/storage/recompressTracked.php uses the data stored in it.

Given that the former script, when run, will delete and recreate the table, can the entire table just be deleted (and recreated when the next recompression run happens)?

(In reply to comment #1)

Given that the former script, when run, will delete and recreate the table,
can the entire table just be deleted (and recreated when the next
recompression run happens)?

Yes. I wanted to keep it around for a while, to allow reconstruction of the old text table in case a data loss bug in recompressTracked.php was discovered, but the source clusters have been deleted now (11, 12, 16, 17, 18, 19), so that justification no longer holds. Also, it's been 3 years since it was last run, with no such bugs being reported.

jcrespo added a project: DBA.
jcrespo subscribed.

Doing a test delete of blob_tracking table on db1073.

Change 448986 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/extensions/WikimediaMaintenance@master] Remove 'fixT159372.php' script

https://gerrit.wikimedia.org/r/448986

Change 449774 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/extensions/WikimediaMaintenance@master] Remove testRctComplete.php script

https://gerrit.wikimedia.org/r/449774

tstarling renamed this task from blob_tracking indexes apparently unused to Drop blob_tracking and blob_orphans everywhere.Aug 8 2018, 6:28 AM
tstarling updated the task description. (Show Details)

Change 449774 abandoned by Krinkle:
Remove testRctComplete.php script

https://gerrit.wikimedia.org/r/449774

Marostegui moved this task from Backlog to In progress on the DBA board.
Marostegui subscribed.

I am going to start getting rid of these tables

Mentioned in SAL (#wikimedia-operations) [2018-08-21T08:37:02Z] <marostegui> Rename blob_tracking and blob_orphans on db1089 - T59186

For now and to really make sure nothing uses it, I have renamed then on db1089 and will give them 24h:

root@db1089.eqiad.wmnet[enwiki]> show tables like 'TO_DROP%';
+-----------------------------+
| Tables_in_enwiki (TO_DROP%) |
+-----------------------------+
| TO_DROP_blob_orphans        |
| TO_DROP_blob_tracking       |
+-----------------------------+
2 rows in set (0.00 sec)

Mentioned in SAL (#wikimedia-operations) [2018-08-23T05:19:25Z] <marostegui> Drop blob_orphans and blob_tracking from s5 T59186

Mentioned in SAL (#wikimedia-operations) [2018-08-23T05:46:59Z] <marostegui> Drop blob_orphans and blob_tracking from s6 T59186

Mentioned in SAL (#wikimedia-operations) [2018-08-23T05:50:24Z] <marostegui> Drop blob_orphans and blob_tracking from s4 T59186

Mentioned in SAL (#wikimedia-operations) [2018-08-23T05:57:49Z] <marostegui> Drop blob_orphans and blob_tracking from s2 T59186

Mentioned in SAL (#wikimedia-operations) [2018-08-23T06:47:51Z] <marostegui> Drop blob_orphans and blob_tracking from s7 T59186

Mentioned in SAL (#wikimedia-operations) [2018-08-23T07:12:03Z] <marostegui> Drop blob_orphans and blob_tracking from s1 T59186

Mentioned in SAL (#wikimedia-operations) [2018-08-23T07:23:38Z] <marostegui> Drop blob_orphans and blob_tracking from s3 on codfw (lag will be generated on codfw) T59186

Marostegui updated the task description. (Show Details)

After almost 5 years...tables gone! :)