Page MenuHomePhabricator
Create Task
Maniphest T127164

Some databases cannot be backed up/replicated on toolsdb
Closed, ResolvedPublic
Actions

Description

The following tables cannot be replicated due to performance concerns (usually it means they are extremely large or they cannot be effectively replicated because they are the result of continuous imports. In order to not affect the rest of the users, replication has been turned off in these cases:

s51412__data.dewiki_templatedata
s51071__templatetiger_p [all tables]

This doesn't create problems on the database itself, but it means they would be lost in the event of a software or hardware failure. The actionable I need from @APPER, @Kolossos and @yuvipanda is one of these:

  • Aknowledge "it is ok, I can recover those tables if I lose them completely"
  • Confirm "it is ok, I have my own backups outside the database and check those regularly"
  • Ask for help "please help me backup those outside of the replication process"- which may require table changes

Related Objects

Event Timeline

jcrespo created this task.Feb 17 2016, 9:32 AM
jcrespo claimed this task.
jcrespo raised the priority of this task from to Needs Triage.
jcrespo updated the task description. (Show Details)
jcrespo added projects: DBA, Cloud-Services.
jcrespo added subscribers: jcrespo, APPER, Kolossos, yuvipanda.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptFeb 17 2016, 9:32 AM
jcrespo added a subscriber: Stigmj.Feb 17 2016, 9:44 AM

I may add here all tables in s52721__pagecount_stats_p. Performance is not a perfect science.

jcrespo moved this task from Triage to In progress on the DBA board.Feb 17 2016, 9:52 AM
Stigmj added a comment.Feb 17 2016, 10:11 AM

Thanks for the notice. My DB's are possible to recover in case of a complete wipeout, but that would take several months to do. What is the issue with my DB's? Are they too big or too busy?

jcrespo added a comment.Feb 17 2016, 10:18 AM

s52721__pagecount_stats_p is currently being replicated, but for some time it was on the verge of being filtered out. Right now I do not see any issue with it, but it showed me bad numbers in the past. Did you use to write heavily to it, but no so much anymore? If that is the case, you can ignore this ticket as no filter will be applied with the current write level.

Note that current redundancy is real-time, so that does not protect against accidental deletions.

Kolossos added a comment.Feb 17 2016, 10:19 PM

I can recreate all data inside templatetiger, in the case of a disaster.

APPER added a comment.Feb 17 2016, 10:46 PM

I can recreate s51412__data.dewiki_templatedata if needed. It will take some time but it would be possible.

jcrespo closed this task as Resolved.Feb 18 2016, 6:22 AM

Thank you all. Please allow me to help with backups if you do not have them for a faster recovery.

jcrespo mentioned this in T150759: s51412__data reverted to last year state.Nov 15 2016, 12:18 PM
• Phabricator_maintenance removed a subscriber: yuvipanda.Jun 7 2017, 6:44 PM
jcrespo mentioned this in T168584: Labsdb* servers need to be rebooted.Jun 23 2017, 7:52 AM
Framawiki subscribed.Jun 7 2018, 9:28 PM
jcrespo mentioned this in T202055: labsdb1004's toolsdb mariadb is lagging behind labsdb1005.Aug 17 2018, 6:26 AM
jcrespo mentioned this in T216441: Evaluate transferring the non-replicated tables to the new toolsdb server.Feb 19 2019, 12:26 PM
bd808 mentioned this in T236101: Find a way to remove non-replicated tables from ToolsDB.Oct 21 2019, 8:10 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits