Page MenuHomePhabricator

Bring labswiki database tables up to date
Closed, ResolvedPublic

Description

It seems labswiki has numerous long defunct (uninstalled extension, or no longer part of MW core) tables, and probably has a few schema drifts too along the way.

tableneededto replicate?comment
abuse_filteryes
abuse_filter_actionyes
abuse_filter_historyyes
abuse_filter_logyes
actoryes
archiveyes
babelyes
betafeatures_user_countsyes
bv2017_editsnodata is presumably no longer relevant
categoryyes
categorylinksyes
change_tagyes
change_tag_defyes
commentyes
configno
contentyes
content_modelsyes
cu_changesyes
cu_logyes
echo_email_batchyes
echo_eventyes
echo_notificationyes
echo_target_pageyes
external_userno
externallinksyes
filearchiveyes
flow_ext_refnodropped
flow_revisionnodropped
flow_subscriptionnodropped
flow_topic_listnodropped
flow_tree_nodenodropped
flow_tree_revisionnodropped
flow_wiki_refnodropped
flow_workflownodropped
geo_tagsShouldn't be needed (GeoData not installed)
global_block_whitelistyesWill be needed when wiki is SUL'd
globalblocksShouldn't be needed (only for "central wiki")
globalimagelinksShouldn't be needed (only for "central wiki")
hiddennodropped
historical_threadLQT
imageyes
imagelinksyes
interwikiyes
ip_changesyes
ipblocksyes
ipblocks_restrictionsyes
iwlinksyes
jobyes
l10n_cacheyes
langlinksyes
ldap_domainsyes
linteryes
log_searchyes
loggingyes
mathoidyes
module_depsyes
msg_resourcenodropped
msg_resource_linksnodropped
oaiauditnodropped
oaiharvestnodropped
oaiusernodropped
oathauth_usersyes
oathauth_users_restoreyes
oauth2_access_tokensyes
oauth_accepted_consumeryes
oauth_registered_consumeryes
objectcacheyes
oldimageyes
openstack_notification_eventnodropped
openstack_puppet_classesnodropped
openstack_puppet_groupsnodropped
openstack_puppet_varsnodropped
openstack_tokensnodropped
pageyes
page_propsyes
page_restrictionsyes
pagelinksyes
protected_titlesyes
querycacheyes
querycache_infoyes
querycachetwoyes
recentchangesyes
redirectyes
revisionyes
revision_actor_tempyesT215466
revision_comment_tempyesT215466
searchindexyes
site_identifiersyes
site_statsyes
sitesyes
slot_rolesyes
slotsyes
spoofuseryes
templatelinksyes
textyes
threadLQT
thread_historyLQT
thread_pending_relationshipLQT
thread_reactionLQT
titlekeynodropped
trackbacksnodropped
transcachenodropped
updatelogyes
updatesnodropped
uploadstashyes
useryes
user_daily_contribsnodropped
user_former_groupsyes
user_groupsyes
user_message_stateLQT
user_newtalkyes
user_openidnodropped
user_propertiesyes
watchlistyes
watchlist_expiryyes
wikilove_logyes

Related Objects

Event Timeline

Reedy updated the task description. (Show Details)
Reedy updated the task description. (Show Details)

Tables marked <dropped> are backed up temporarily on db1128 in /home/andrew/wikitechtables

Mentioned in SAL (#wikimedia-operations) [2021-06-01T21:09:49Z] <andrewbogott> dropping a bunch of tables from the labswiki db as per T284108

Andrew updated the task description. (Show Details)
Reedy updated the task description. (Show Details)

I know @Ladsgroup has filled some tasks regarding schema drifts on wikitech: T269348

btw I noticed this during the change, the actor table is empty, you need to run its migration script I think.

btw I noticed this during the change, the actor table is empty, you need to run its migration script I think.

?

MariaDB [labswiki]> select count(*) from actor;
+----------+
| count(*) |
+----------+
|    25902 |
+----------+
1 row in set (0.015 sec)

oh my bad. It was rows affected and it was zero because I already set actor to binary charset. Sorry.

It's my understanding that this data is now ready for T167973 -- please lmk if anyone disagrees, otherwise I think we can close this.

So, it is fine to proceed with the normal filtering we have for the other wikis or do we need extra ones?:
These are the tables we currently ignore:

replicate-wild-ignore-table = %.__wmf_checksums
replicate-wild-ignore-table = %.accountaudit_login
replicate-wild-ignore-table = %.arbcom1_vote
replicate-wild-ignore-table = %.archive_old
replicate-wild-ignore-table = %.blob_orphans
replicate-wild-ignore-table = %.blob_tracking
replicate-wild-ignore-table = %.bot_passwords
replicate-wild-ignore-table = %.bv2009_edits
replicate-wild-ignore-table = %.categorylinks_old
replicate-wild-ignore-table = %.click_tracking
replicate-wild-ignore-table = %.cu_changes
replicate-wild-ignore-table = %.cu_log
replicate-wild-ignore-table = %.cur
replicate-wild-ignore-table = %.discussiontools_subscription
replicate-wild-ignore-table = %.echo_email_batch
replicate-wild-ignore-table = %.echo_event
replicate-wild-ignore-table = %.echo_target_page
replicate-wild-ignore-table = %.echo_unread_wikis
replicate-wild-ignore-table = %.echo_notification
replicate-wild-ignore-table = %.echo_push_subscription
replicate-wild-ignore-table = %.edit_page_tracking
replicate-wild-ignore-table = %.email_capture
replicate-wild-ignore-table = %.exarchive
replicate-wild-ignore-table = %.exrevision
replicate-wild-ignore-table = %.globalnames
replicate-wild-ignore-table = %.growthexperiments_link_recommendations
replicate-wild-ignore-table = %.growthexperiments_link_submissions
replicate-wild-ignore-table = %.growthexperiments_mentor_mentee
replicate-wild-ignore-table = %.growthexperiments_mentee_data
replicate-wild-ignore-table = %.hidden
replicate-wild-ignore-table = %.image_old
replicate-wild-ignore-table = %.job
replicate-wild-ignore-table = %.linkscc
replicate-wild-ignore-table = %.localnames
replicate-wild-ignore-table = %.log_search
replicate-wild-ignore-table = %.logging_old
replicate-wild-ignore-table = %.long_run_profiling
replicate-wild-ignore-table = %.migrateuser_medium
replicate-wild-ignore-table = %.moodbar_feedback
replicate-wild-ignore-table = %.moodbar_feedback_response
replicate-wild-ignore-table = %.msg_resource
replicate-wild-ignore-table = %.oathauth_users
replicate-wild-ignore-table = %.oauth_accepted_consumer
replicate-wild-ignore-table = %.oauth_ratelimit_client_tier
replicate-wild-ignore-table = %.oauth_registered_consumer
replicate-wild-ignore-table = %.oauth2_access_tokens
replicate-wild-ignore-table = %.objectcache
replicate-wild-ignore-table = %.old_growth
replicate-wild-ignore-table = %.oldimage_old
replicate-wild-ignore-table = %.optin_survey
replicate-wild-ignore-table = %.prefstats
replicate-wild-ignore-table = %.prefswitch_survey
replicate-wild-ignore-table = %.profiling
replicate-wild-ignore-table = %.querycache
replicate-wild-ignore-table = %.querycache_info
replicate-wild-ignore-table = %.querycache_old
replicate-wild-ignore-table = %.querycachetwo
replicate-wild-ignore-table = %.reading_list
replicate-wild-ignore-table = %.reading_list_entry
replicate-wild-ignore-table = %.securepoll_cookie_match
replicate-wild-ignore-table = %.securepoll_elections
replicate-wild-ignore-table = %.securepoll_entity
replicate-wild-ignore-table = %.securepoll_lists
replicate-wild-ignore-table = %.securepoll_msgs
replicate-wild-ignore-table = %.securepoll_options
replicate-wild-ignore-table = %.securepoll_properties
replicate-wild-ignore-table = %.securepoll_questions
replicate-wild-ignore-table = %.securepoll_strike
replicate-wild-ignore-table = %.securepoll_voters
replicate-wild-ignore-table = %.securepoll_votes
replicate-wild-ignore-table = %.spoofuser
replicate-wild-ignore-table = %.text
replicate-wild-ignore-table = %.titlekey
replicate-wild-ignore-table = %.transcache
replicate-wild-ignore-table = %.translate_cache
replicate-wild-ignore-table = %.uploadstash
replicate-wild-ignore-table = %.urlshortcodes
replicate-wild-ignore-table = %.user_newtalk
replicate-wild-ignore-table = %.vote_log
replicate-wild-ignore-table = %.watchlist
replicate-wild-ignore-table = %.watchlist_expiry
replicate-wild-ignore-table = %.wikimedia_editor_tasks_counts
replicate-wild-ignore-table = %.wikimedia_editor_tasks_keys
replicate-wild-ignore-table = %.wikimedia_editor_tasks_targets_passed

ldap_domains should be the only non standard table

I'm struggling to see any reason it should be replicated

CREATE TABLE /*_*/ldap_domains (
  domain_id INT NOT NULL,
  domain VARCHAR(255) NOT NULL,
  user_id INT NOT NULL,
  INDEX user_id (user_id),
  PRIMARY KEY(domain_id)
) /*$wgDBTableOptions*/;

bv2017_edits shouldn't be either, and I suspect can just be dropped anyway (what do we do about the board vote tables on other wikis?)

ldap_domains has been excluded from replication: T284106#7142455

I'm about to close this unless someone asks me not to :)