Page MenuHomePhabricator
Create Task
Maniphest T260112

Remove muswiki and mhwiktionary from s3
Closed, ResolvedPublic
Actions

Description

muswiki and mhwiktionary are two closed wikis that were moved from s3 to s5 at T259004.
We need to get them removed from s3.

Let's first rename the tables on the master and leave it like that for a few days before actually going ahead and dropping the databases.
It needs to be done without replication

Related Objects
Search...

Event Timeline

Marostegui created this task.Aug 11 2020, 8:59 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 11 2020, 8:59 AM
Marostegui triaged this task as Medium priority.Aug 11 2020, 8:59 AM
Marostegui moved this task from Triage to In progress on the DBA board.
Marostegui added a subtask: T259004: Move muswiki and mhwiktionary (closed wikis) from s3 to s5.
Stashbot added a comment.Aug 11 2020, 9:11 AM

Mentioned in SAL (#wikimedia-operations) [2020-08-11T09:11:28Z] <marostegui> Rename tables on muswiki and mhwiktionary on s3 master (db1123) without replication T260112

Marostegui added a comment.Aug 11 2020, 9:13 AM

I have renamed the tables on s3 master only (db1123) for muswiki and mhwiktionary.
If any writes attempts to happen, it will fail. Let's leave this for a few days while we monitor logstash for any issues.

This is how they look like now:

# mysql.py -hdb1123 -e "show tables" muswiki
+----------------------------------+
| Tables_in_muswiki                |
+----------------------------------+
| T260112_abuse_filter             |
| T260112_abuse_filter_action      |
| T260112_abuse_filter_history     |
| T260112_abuse_filter_log         |
| T260112_actor                    |
| T260112_archive                  |
| T260112_babel                    |
| T260112_betafeatures_user_counts |
| T260112_blobs                    |
| T260112_bv2009_edits             |
| T260112_bv2011_edits             |
| T260112_bv2013_edits             |
| T260112_bv2015_edits             |
| T260112_bv2017_edits             |
| T260112_category                 |
| T260112_categorylinks            |
| T260112_change_tag               |
| T260112_change_tag_def           |
| T260112_comment                  |
| T260112_content                  |
| T260112_content_models           |
| T260112_cu_changes               |
| T260112_cu_log                   |
| T260112_cur                      |
| T260112_externallinks            |
| T260112_filearchive              |
| T260112_geo_tags                 |
| T260112_global_block_whitelist   |
| T260112_hidden                   |
| T260112_image                    |
| T260112_imagelinks               |
| T260112_interwiki                |
| T260112_ip_changes               |
| T260112_ipblocks                 |
| T260112_ipblocks_restrictions    |
| T260112_iwlinks                  |
| T260112_job                      |
| T260112_l10n_cache               |
| T260112_langlinks                |
| T260112_linter                   |
| T260112_log_search               |
| T260112_logging                  |
| T260112_mathoid                  |
| T260112_module_deps              |
| T260112_objectcache              |
| T260112_oldimage                 |
| T260112_page                     |
| T260112_page_props               |
| T260112_page_restrictions        |
| T260112_pagelinks                |
| T260112_pif_edits                |
| T260112_povwatch_log             |
| T260112_povwatch_subscribers     |
| T260112_protected_titles         |
| T260112_querycache               |
| T260112_querycache_info          |
| T260112_querycachetwo            |
| T260112_recentchanges            |
| T260112_redirect                 |
| T260112_revision                 |
| T260112_revision_actor_temp      |
| T260112_revision_comment_temp    |
| T260112_searchindex              |
| T260112_securepoll_cookie_match  |
| T260112_securepoll_elections     |
| T260112_securepoll_entity        |
| T260112_securepoll_lists         |
| T260112_securepoll_msgs          |
| T260112_securepoll_options       |
| T260112_securepoll_properties    |
| T260112_securepoll_questions     |
| T260112_securepoll_strike        |
| T260112_securepoll_voters        |
| T260112_securepoll_votes         |
| T260112_site_identifiers         |
| T260112_site_stats               |
| T260112_sites                    |
| T260112_slot_roles               |
| T260112_slots                    |
| T260112_spoofuser                |
| T260112_templatelinks            |
| T260112_text                     |
| T260112_transcache               |
| T260112_transcode                |
| T260112_updatelog                |
| T260112_uploadstash              |
| T260112_user                     |
| T260112_user_former_groups       |
| T260112_user_groups              |
| T260112_user_newtalk             |
| T260112_user_properties          |
| T260112_watchlist                |
| T260112_watchlist_expiry         |
| T260112_wbc_entity_usage         |
+----------------------------------+

# mysql.py -hdb1123 -e "show tables" mhwiktionary
+----------------------------------+
| Tables_in_mhwiktionary           |
+----------------------------------+
| T260112_abuse_filter             |
| T260112_abuse_filter_action      |
| T260112_abuse_filter_history     |
| T260112_abuse_filter_log         |
| T260112_actor                    |
| T260112_archive                  |
| T260112_babel                    |
| T260112_betafeatures_user_counts |
| T260112_blobs                    |
| T260112_bv2009_edits             |
| T260112_bv2011_edits             |
| T260112_bv2013_edits             |
| T260112_bv2015_edits             |
| T260112_bv2017_edits             |
| T260112_category                 |
| T260112_categorylinks            |
| T260112_change_tag               |
| T260112_change_tag_def           |
| T260112_comment                  |
| T260112_content                  |
| T260112_content_models           |
| T260112_cu_changes               |
| T260112_cu_log                   |
| T260112_cur                      |
| T260112_externallinks            |
| T260112_filearchive              |
| T260112_geo_tags                 |
| T260112_global_block_whitelist   |
| T260112_hidden                   |
| T260112_image                    |
| T260112_imagelinks               |
| T260112_interwiki                |
| T260112_ip_changes               |
| T260112_ipblocks                 |
| T260112_ipblocks_restrictions    |
| T260112_iwlinks                  |
| T260112_job                      |
| T260112_l10n_cache               |
| T260112_langlinks                |
| T260112_linter                   |
| T260112_log_search               |
| T260112_logging                  |
| T260112_mathoid                  |
| T260112_module_deps              |
| T260112_objectcache              |
| T260112_oldimage                 |
| T260112_page                     |
| T260112_page_props               |
| T260112_page_restrictions        |
| T260112_pagelinks                |
| T260112_pif_edits                |
| T260112_povwatch_log             |
| T260112_povwatch_subscribers     |
| T260112_protected_titles         |
| T260112_querycache               |
| T260112_querycache_info          |
| T260112_querycachetwo            |
| T260112_recentchanges            |
| T260112_redirect                 |
| T260112_revision                 |
| T260112_revision_actor_temp      |
| T260112_revision_comment_temp    |
| T260112_searchindex              |
| T260112_securepoll_cookie_match  |
| T260112_securepoll_elections     |
| T260112_securepoll_entity        |
| T260112_securepoll_lists         |
| T260112_securepoll_msgs          |
| T260112_securepoll_options       |
| T260112_securepoll_properties    |
| T260112_securepoll_questions     |
| T260112_securepoll_strike        |
| T260112_securepoll_voters        |
| T260112_securepoll_votes         |
| T260112_site_identifiers         |
| T260112_site_stats               |
| T260112_sites                    |
| T260112_slot_roles               |
| T260112_slots                    |
| T260112_spoofuser                |
| T260112_templatelinks            |
| T260112_text                     |
| T260112_transcache               |
| T260112_transcode                |
| T260112_updatelog                |
| T260112_uploadstash              |
| T260112_user                     |
| T260112_user_former_groups       |
| T260112_user_groups              |
| T260112_user_newtalk             |
| T260112_user_properties          |
| T260112_watchlist                |
| T260112_watchlist_expiry         |
| T260112_wbc_entity_usage         |
+----------------------------------+
Marostegui mentioned this in T238966: Apply updates for MCR, actor migration, and content migration, to production wikis..Aug 13 2020, 7:29 AM
Marostegui added a comment.Aug 13 2020, 11:51 AM

No errors so far, for the last 2 days.
If all continues like this, on Monday I will remove the tables and the database from s3.

Bugreporter mentioned this in T246055: Drop DB tables for now-deleted fixcopyrightwiki from production.Sep 10 2020, 3:14 PM
LSobanski added a project: Data-Persistence.Oct 8 2020, 9:45 AM
LSobanski moved this task from In progress to Ready on the DBA board.Oct 8 2020, 9:55 AM
Urbanecm added a comment.Oct 8 2020, 11:07 AM
In T260112#6382513, @Marostegui wrote:

No errors so far, for the last 2 days.
If all continues like this, on Monday I will remove the tables and the database from s3.

Did that happen? :-)

Marostegui added a comment.Oct 8 2020, 11:07 AM

No, I am waiting for eqiad to be back as active

LSobanski removed a project: Data-Persistence.Oct 28 2020, 2:06 PM
Marostegui added a comment.Oct 30 2020, 6:43 AM

Just to confirm, this can now happen and wikis, as expected, have no writes on s3.

root@db1075:/srv/sqldata/muswiki# ls -lhrt *.ibd | tail -n 2
-rw-rw---- 1 mysql mysql 9.0M Aug 11 08:32 logging.ibd
-rw-rw---- 1 mysql mysql 512K Aug 11 09:49 user_properties.ibd

root@db1075:/srv/sqldata/mhwiktionary# ls -lhrt *.ibd | tail -n 2
-rw-rw---- 1 mysql mysql 176K Aug 11 09:49 cu_changes.ibd
-rw-rw---- 1 mysql mysql 9.0M Aug 11 09:49 logging.ibd

We need to drop the databases without replication on each host.

Marostegui moved this task from Ready to In progress on the DBA board.EditedNov 4 2020, 8:15 AM

Deletion progress:
NOT NEEDED labsdb1012
NOT NEEDED labsdb1011
NOT NEEDED labsdb1010
NOT NEEDED labsdb1009

  • dbstore1004:3313
  • db2127
  • db2109
  • db2105
  • db2098:3313
  • db2094:3313
  • db2074
  • db1124:3313
  • db1123
  • db1112
  • db1095
  • db1078
  • db1075
Marostegui added a comment.Nov 4 2020, 8:32 AM

This has been cleaned up on codfw.
Waiting a few hours before going for eqiad.

Urbanecm added a comment.Nov 5 2020, 2:35 AM

Thanks Marostegui!

Marostegui closed this task as Resolved.Nov 5 2020, 10:23 AM

This is done:

# ./section s3 | while read host port; do echo "$host:$port"; mysql.py -h$host:$port -e "show databases like 'muswiki'; show databases like 'mhwiktionary'";done
labsdb1012.eqiad.wmnet:3306
Database (muswiki)
muswiki
Database (mhwiktionary)
mhwiktionary
labsdb1011.eqiad.wmnet:3306
Database (muswiki)
muswiki
Database (mhwiktionary)
mhwiktionary
labsdb1010.eqiad.wmnet:3306
Database (muswiki)
muswiki
Database (mhwiktionary)
mhwiktionary
labsdb1009.eqiad.wmnet:3306
Database (muswiki)
muswiki
Database (mhwiktionary)
mhwiktionary
dbstore1004.eqiad.wmnet:3313
db2127.codfw.wmnet:3306
db2109.codfw.wmnet:3306
db2105.codfw.wmnet:3306
db2098.codfw.wmnet:3313
db2094.codfw.wmnet:3313
db2074.codfw.wmnet:3306
db1124.eqiad.wmnet:3313
db1123.eqiad.wmnet:3306
db1112.eqiad.wmnet:3306
db1095.eqiad.wmnet:3313
db1078.eqiad.wmnet:3306
db1075.eqiad.wmnet:3306
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL