run pt-tablechecksum on s6
Open, Needs TriagePublic

Description

The last database of s2 is being checksummed on T154485 so it is time to move on to another shard.

s6 has the following servers that need to be decommissioned: db1023,db1022,db1030,db1037.
Let's checksum their data and see if we can decommission them easily

This will be a good test to run p-t-c on slightly larger tables than s2, which so far, gave no problem, but the largest table was a 40G revision table on itwiki.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptWed, Mar 15, 10:33 AM
Marostegui moved this task from Triage to Next on the DBA board.Wed, Mar 15, 10:33 AM

The following tables per database do not have a PK and will need to be excluded:

frwiki:

archive_save
categorylinks
change_tag
click_tracking
click_tracking_user_properties
cur
edit_page_tracking
flaggedrevs_tracking
hidden
imagelinks
interwiki
iwlinks
l10n_cache
langlinks
linkscc
localisation_file_hash
log_search
logging_pre_1_10
math
module_deps
msg_resource
msg_resource_links
objectcache
oldimage
pagelinks
prefstats
prefswitch_survey
profiling
querycache
querycache_info
querycachetwo
searchindex
securepoll_lists
securepoll_msgs
securepoll_properties
site_identifiers
site_stats
tag_summary
templatelinks
text
transcache
user_former_groups
user_newtalk
user_properties
watchlist

jawiki:

categorylinks
change_tag
click_tracking
click_tracking_user_properties
cur
edit_page_tracking
hidden
imagelinks
interwiki
iwlinks
l10n_cache
langlinks
linkscc
localisation_file_hash
log_search
logging_pre_1_10
math
module_deps
msg_resource
msg_resource_links
objectcache
oldimage
pagelinks
prefstats
prefswitch_survey
querycache
querycache_info
querycachetwo
searchindex
securepoll_lists
securepoll_msgs
securepoll_properties
site_identifiers
site_stats
tag_summary
templatelinks
text
transcache
user_former_groups
user_newtalk
user_properties
watchlist

ruwiki:

categorylinks
change_tag
click_tracking
click_tracking_user_properties
cur
edit_page_tracking
ep_users_per_course
hidden
imagelinks
interwiki
iwlinks
l10n_cache
langlinks
linkscc
log_search
logging_pre_1_10
math
module_deps
msg_resource
msg_resource_links
objectcache
oldimage
pagelinks
prefstats
prefswitch_survey
querycache
querycache_info
querycachetwo
searchindex
securepoll_lists
securepoll_msgs
securepoll_properties
site_identifiers
site_stats
tag_summary
templatelinks
text
transcache
user_former_groups
user_newtalk
user_properties
watchlist

dsns_s6 table looks good, and I have added the replication filter to the rc slaves (db1037 and db2039) to ignore the wmf_checksums table to avoid the issue we already had: T154485#3050716

Replicate_Wild_Ignore_Table: ops.__wmf_checksums

Mentioned in SAL (#wikimedia-operations) [2017-03-16T07:08:48Z] <marostegui> Starting pt-table-checksum on s6 (frwiki) - T160509

I have just started the check on frwiki.

Finished running pt-table-checksum on frwiki.
Differences found on:

Differences on db1030
frwiki.archive
Differences on dbstore1002
frwiki.archive
frwiki.page_props
frwiki.wbc_entity_usage

Mentioned in SAL (#wikimedia-operations) [2017-03-21T07:22:16Z] <marostegui> Run pt-table-checksum on s6 (jawiki) - T160509

jawiki has been finished and there are no differences! \o/

Mentioned in SAL (#wikimedia-operations) [2017-03-21T10:33:07Z] <marostegui> Run pt-table-checksum on s6 (ruwiki) - https://phabricator.wikimedia.org/T160509

Marostegui moved this task from Next to In progress on the DBA board.Tue, Mar 21, 3:21 PM

ruwiki finished too, so the whole shard has been checksumed.
ruwiki has a few differences:

Differences on db2046
TABLE CHUNK CNT_DIFF CRC_DIFF CHUNK_INDEX LOWER_BOUNDARY UPPER_BOUNDARY
ruwiki.geo_tags 1 0 1 PRIMARY 1 81696384

Differences on db1030
TABLE CHUNK CNT_DIFF CRC_DIFF CHUNK_INDEX LOWER_BOUNDARY UPPER_BOUNDARY
ruwiki.archive 51330 0 1 PRIMARY 6612569 6612675
ruwiki.archive 54562 0 1 PRIMARY 7105225 7105398
ruwiki.archive 54563 0 1 PRIMARY 7105399 7105510

And dbstore1002 has a lot more.