Page MenuHomePhabricator

Database tables to be dropped on Wikimedia wikis and other WMF databases (tracking)
Open, NormalPublic

Description

This bug tracks obsolete or otherwise unneeded database tables on Wikimedia wikis that should be dropped.


URL: https://wikitech.wikimedia.org/wiki/Database_tables
See Also: T18660


TableExtension/SourceRecom. contactStatusTaskNotes
__wmf_checksums.ibdpt-table-checksumDBATo removeBut they are created by DBAs every time data checks run
_image_new.ibdpt-online-schema-change going wrongDBARemovedT171429probably an alter table abandoned, drop where it exists
arbcom1_vote.ibdE:BoardVote@tstarlingKeepData to be archived. Once done, this table can be dropped.
blob_orphans.ibd blob_tracking.ibdtrackBlobs.php@tstarlingRemovedT59186"Tracking table for blob rows that aren't tracked by the text table" related to blob_tracking
bv2009_edits.ibdE:SecurePoll@tstarlingTo removeThese bv_*edits tables are intermediate data (edit counts) generated by SecurePoll CLI scripts. The voter lists generated from them are stored in securepoll_lists. Probably not worth archiving since they were generated from archived edit data.
bv2011_edits.ibdE:SecurePoll@tstarlingTo remove
bv2015_edits.ibdE:SecurePoll@tstarlingTo remove
click_tracking.ibdE:Click_Tracking?RemovedT115982Dep. of E:ArticleFeedback
click_tracking_events.ibdE:Click_Tracking?RemovedT115982Dep. of E:ArticleFeedback
click_tracking_user_propertiesE:Click_Tracking?RemovedT115982Dep. of E:ArticleFeedback
cur.ibdMW Core@tstarlingKeepNeed to audit text, ar_text for HistoryBlobCurStub objects
edit_page_tracking.ibdE:EditPageTrackingRemovedT57385
email_capture.ibdE:Email_Capture?RemovedT57676Dep. of E:ArticleFeedback
ep_cas.ibdE:EducationProgram@JeroenDeDauwKeepActively deployed
ep_instructors.ibdE:EducationProgram@JeroenDeDauwKeepActively deployed
ep_oas.ibdE:EducationProgram@JeroenDeDauwKeepActively deployed
ep_revisions.ibdE:EducationProgram@JeroenDeDauwKeepActively deployed
exarchive.ibdMW Core??(old copy of archive table?)
exrevision.ibdMW Core??(old copy of revision table?)
filejournal.ibdMW Core@aaronaaron committed this back in rSVN113704: [FileBackend] - I guess Wikimedia isn't using it?
flaggedimages.ibdE:FlaggedRevs@aaron
flaggedrevs_stats.ibdE:FlaggedRevs@aaron
flaggedrevs_stats2.ibdE:FlaggedRevs@aaron
flaggedtemplates.ibdE:FlaggedRevs@aaron
global_block_whitelist.ibdE:GlobalBlocking@Legoktm/@GlaisherKeepActively used
hidden.ibdE:Oversight@Krenair?This should theoretically be obsoleted by the removal of the Oversight extension in T34628, but let me double check.
interwiki.ibdMW Core (not used by WMF)KeepT169376Just truncate. (done)
job.ibdMW Core (not used by WMF)KeepT169377Just truncate. (done)
l10n_cache.ibdMW Core (not used WMF deployment)KeepT169375Just truncate. (done)
linkscc.ibdMW Core (removed in 1.5)@tstarlingRemovedT192056
localisation.ibdExtension:LocalisationUpdate (table removed)RemovedT119811
localisation_file_hash.ibdExtension:LocalisationUpdate (table removed)RemovedT119811
long_run_profiling.ibdMW Core@tstarlingRemovedT194661@tstarling made this as a temporary copy of the profiling table
mark_as_helpful.ibdE:MarkAsHelpfulRemovedT151655
mathoid.ibdE:Math@PhysikerweltTo removeNow stored in Restbase T74240: Remove PNG only rendering mode
moodbar_feedback.ibdE:MoodBarRemovedT153033Undeployed from WMF
moodbar_feedback_responseE:MoodBarRemovedT153033Undeployed from WMF
msg_resource.ibdMW Core (removed in 1.27)KrinkleRemovedT194663Removed from core per T113092
msg_resource_links.ibdMW Core (removed in 1.27)KrinkleRemovedT194663Removed from core per T113092
old_growth.ibdRemovedT115982
pagetriage_tags.ibdE:PageTriageKeepAppears to be used, but only for look-ups. Could be migrated to a data/config file.
pif_edits.ibd?
povwatch_log.ibdE:PovWatchRemovedT54924
povwatch_subscribers.ibdE:PovWatchRemovedT54924
pr_index.ibdE:ProofreadPageKeepTable may exist on wikis that don't use Extension. Can be removed from those
prefstats.ibdE:PrefStatsRemovedT154490
prefswitch_survey.ibdE:PrefSwitchRemovedT173439
reader_feedbackE:ReaderFeedback@ReedyRemovedT174586
reader_feedback_historyE:ReaderFeedback@ReedyRemovedT174586
reader_feedback_pagesE:ReaderFeedback@ReedyRemovedT174586
searchindex.ibdMW Core (unused by WMF)@demonKeepT167978Truncate only. (done)
securepoll_cookie_match.ibdE:SecurePoll@tstarlingKeep
securepoll_elections.ibdE:SecurePoll@tstarlingKeep
securepoll_entity.ibdE:SecurePoll@tstarlingKeep
securepoll_lists.ibdE:SecurePoll@tstarlingKeep
securepoll_msgs.ibdE:SecurePoll@tstarlingKeep
securepoll_options.ibdE:SecurePoll@tstarlingKeep
securepoll_properties.ibdE:SecurePoll@tstarlingKeep
securepoll_questions.ibdE:SecurePoll@tstarlingKeep
securepoll_strike.ibdE:SecurePoll@tstarlingKeep
securepoll_votes.ibdE:SecurePoll@tstarlingKeep
site_identifiers.ibdMW CoreKeepActively in use
sites.ibdMW CoreKeepActively in use
tag_summary.ibdMW Core@LadsgroupTo removeT209525Deprecated, use change_tag instead
transcache.ibdMW Core (not used in WMF)KeepCore table, just truncate
transcode.ibdE:TimedMediaHandler@brionKeepActively used
updatelog.ibdMW Core (not used in WMF)@demonKeepT174804Core table, just truncate
valid_tag.ibdMW Core@LadsgroupRemovedT212254This is in use by MW core for change tagging
vote_log.ibdE:BoardVote@tstarlingKeepData will be archived, once done, it may be dropped.

Details

Reference
bz52921

Related Objects

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.
StatusAssignedTask
OpenNone
OpenNone
ResolvedMarostegui
Resolvedjcrespo
ResolvedMarostegui
ResolvedMarostegui
ResolvedSpringle
ResolvedMarostegui
ResolvedSpringle
ResolvedSpringle
ResolvedSpringle
ResolvedMarostegui
Resolvedjcrespo
ResolvedMarostegui
ResolvedMarostegui
Resolvedjcrespo
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
Resolvedjcrespo
Resolvedjcrespo
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
DeclinedNone
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
DuplicateNone
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
Resolvedelukey
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
OpenNone
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui
ResolvedMarostegui

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Krinkle updated the task description. (Show Details)Mar 28 2018, 12:15 AM
Krinkle updated the task description. (Show Details)Mar 28 2018, 12:21 AM
Krinkle updated the task description. (Show Details)
Krinkle updated the task description. (Show Details)Mar 28 2018, 12:25 AM
Krinkle updated the task description. (Show Details)

Since the "Removable" table column is not sortable, perhaps splitting the table into multiple tables would be useful? The current mix of removed, unremovable, and pending removal tables is somewhat confusing to digest.

@MZMcBride I suggested moving it to mediawiki.org or wikitech, help is welcome!

Krinkle updated the task description. (Show Details)Apr 2 2018, 12:53 PM

I've attempted to make the table a bit more readable by shortening the labels a bit, and by rephrasing "Removable: Yes, No, Removed" to "Status: To remove, Keep, Removed".

jcrespo updated the task description. (Show Details)Apr 2 2018, 1:03 PM

Hey, @Marostegui, when deleting tables, can you check views on labs, too? I think there are some to be deleted that complains on mysql_upgrade (see logs). Or we can ask cloud team to help with that clean up.

Hey, @Marostegui, when deleting tables, can you check views on labs, too? I think there are some to be deleted that complains on mysql_upgrade (see logs). Or we can ask cloud team to help with that clean up.

Sure! I will review the recent tables I have dropped and create a task for Cloud team if views are found!
Thanks for the reminder

Krinkle updated the task description. (Show Details)Apr 9 2018, 6:49 PM
Krinkle updated the task description. (Show Details)Apr 9 2018, 6:52 PM
Marostegui updated the task description. (Show Details)
Marostegui updated the task description. (Show Details)
Marostegui updated the task description. (Show Details)
Marostegui closed subtask Restricted Task as Resolved.Apr 25 2018, 8:39 AM
Marostegui closed subtask Restricted Task as Resolved.Apr 26 2018, 9:59 AM
Marostegui updated the task description. (Show Details)
Teles removed a subscriber: Teles.Apr 27 2018, 2:35 PM

With regards to one of the tables mentioned in the task description (arbcom1_vote.ibd) please note that Extension:BoardVote is marked as archived (https://www.mediawiki.org/wiki/Extension:BoardVote). As far as I can see the project was never moved to Gerrit either. As such, this table can go if not already (haven't checked).

Krinkle updated the task description. (Show Details)May 16 2018, 10:28 PM
Krinkle updated the task description. (Show Details)
Marostegui updated the task description. (Show Details)Aug 23 2018, 10:19 AM

@tstarling As a background task for myself I am slowly cleaning up all the tables listed here to be removed, so they don't pile up much.

I was checking this comment

bv2011_edits.ibd	E:SecurePoll	@tstarling	Yes		
bv2015_edits.ibd	E:SecurePoll	@tstarling	Yes

Those can be removed but they do not have a task for them, if you double confirm they can be killed, I will create a task for those and link them here.

They can be removed.

With regards to one of the tables mentioned in the task description (arbcom1_vote.ibd) please note that Extension:BoardVote is marked as archived (https://www.mediawiki.org/wiki/Extension:BoardVote). As far as I can see the project was never moved to Gerrit either. As such, this table can go if not already (haven't checked).

No, I said archive the data in the table, not the extension source code. Why would archiving the source code help?

My proposal was to export the vote and arbcom1_vote tables to an SQL file and to retain that file indefinitely.

tstarling updated the task description. (Show Details)Nov 21 2018, 5:22 AM
jcrespo updated the task description. (Show Details)Nov 21 2018, 11:09 AM
Anomie changed the status of subtask T209591: Drop table image_comment_temp on all wikis from Stalled to Open.Dec 3 2018, 2:56 PM

@Krenair I was talking with @MarcoAurelio about the hidden table on eswikibooks and we saw it is listed here - can it be dropped everywhere?

Krenair added a comment.EditedDec 13 2018, 10:31 AM

@Krenair I was talking with @MarcoAurelio about the hidden table on eswikibooks and we saw it is listed here - can it be dropped everywhere?

I am no longer in a position to look into this. If I remember correctly it should be, but I'm not able to check.

Krenair updated the task description. (Show Details)Dec 13 2018, 10:31 AM
Marostegui changed the status of subtask T212255: Drop tag_summary table from Open to Stalled.Dec 20 2018, 12:19 PM
Ladsgroup changed the status of subtask T212255: Drop tag_summary table from Stalled to Open.Jan 2 2019, 4:10 PM
Marostegui updated the task description. (Show Details)Jan 11 2019, 6:47 PM
Marostegui added a subscriber: Ladsgroup.
Ladsgroup updated the task description. (Show Details)Jan 11 2019, 7:19 PM