Page MenuHomePhabricator

audit public tables and make sure we dump them all
Open, NormalPublic0 Story Points

Description

Purely internal tables that just connect one foreign key to another are not needed but let's double check that everything else is dumped. It's been awhile.

  • P8634 lists tables broken out into groups of: dumped already, known private, cache/temp, empty on WMF, for review, and one group from which the relevant fields are dumped as xml.
  • P8774 has a smaller set of tables that aren't on all wikis.

Event Timeline

ArielGlenn triaged this task as Normal priority.Jun 20 2019, 10:58 AM
ArielGlenn created this task.
ArielGlenn updated the task description. (Show Details)Jun 20 2019, 12:28 PM
ArielGlenn moved this task from Backlog to Active on the Dumps-Generation board.

We might be able to dump user_former_groups, change_tag_def and babel without privacy implications. Adding @Reedy for comment. (Feel free to remove yourself and/or redirect if you're not the right person to ping.)

Reedy added a comment.Jul 8 2019, 9:32 AM

All three are replicated 1:1 to labs, so I think they're fine to just be dumped too (as someone with labs access can do the same thing)

https://github.com/wikimedia/puppet/blob/eea5e40f9388c3240cee3344f31edcbe71a023a3/modules/profile/templates/labs/db/views/maintain-views.yaml

T103011 gives some nice background on the labs-replicated databases. so adding that here.

Reedy, thanks for the comment, I'll add a patch to dump those tables starting with the next run,

Change 521565 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/dumps@master] add a few more public sql tables to default list to be dumped

https://gerrit.wikimedia.org/r/521565

Change 521566 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/puppet@production] add more public sql tables to xml/sql dumps

https://gerrit.wikimedia.org/r/521566

Change 521565 merged by ArielGlenn:
[operations/dumps@master] add a few more public sql tables to default list to be dumped

https://gerrit.wikimedia.org/r/521565

Change 524438 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/dumps@master] add a few more public sql tables to default list to be dumped

https://gerrit.wikimedia.org/r/524438

Change 524438 merged by ArielGlenn:
[operations/dumps@master] add a few more public sql tables to default list to be dumped

https://gerrit.wikimedia.org/r/524438

Change 521566 merged by ArielGlenn:
[operations/puppet@production] add more public sql tables to xml/sql dumps

https://gerrit.wikimedia.org/r/521566

ArielGlenn updated the task description. (Show Details)Jul 19 2019, 9:04 AM

Change 527505 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/puppet@production] add more public tables for xml/sql dumps

https://gerrit.wikimedia.org/r/527505

@aaron This revision rEFLR848ef073fa89036c40c440016a8092690ddcf56b for FlaggedRevs seems to indicate that flaggedrevs_stats and flaggedrevs_stats2 are no longer used. Do you know or can you point me to someone who could verify that this is the case? If they aren't used, I will add them to my 'don't ever dump these' list. Thanks!

aaron added a comment.Aug 9 2019, 5:54 AM

They were obsoleted by flaggedrevs_statistics.

Thanks a lot! I've updated the patch above to remove those entries. Now just waiting on the wb_terms migration to get further along.