Page MenuHomePhabricator

Review special replica partitioning of certain tables by `xx_user`
Open, Needs TriagePublic

Description

I came across the fact today that the logging table is partitioned by log_user on certain replicas. Reasonably soon that column will be dropped as part of T188327: Deploy refactored actor storage.

As of this morning we shouldn't have any queries reading log_user anymore (they're using log_actor instead), and the writes will be being turned off over the next few weeks (leaving the column with its default 0 in new rows). If the turning off of the writes goes well, the task for dropping log_user will hopefully be filed in early June.

I see we have paritioning for revision and rev_user on some replicas as well. That table is in the a worse situation with respect to reads (queries formerly targeting rev_user are now targeting the revision_actor_temp table) and is the same for writes. The final dropping of rev_user will not be in June; I have no timeframe for T215466 yet.

If we have other core tables partitioned by an xx_user column, they're likely in the same situation as logging.

I have no idea whether we should partition by something else (log_actor?) or just remove the partitioning. The reasoning for separate replicas in 085e3563ed may still apply even without the partitioning.

Note this task doesn't necessarily require that anything be done now. I just wanted to raise the issue with some lead time before the schema change request.

Event Timeline

Anomie created this task.May 13 2019, 5:50 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 13 2019, 5:50 PM
Marostegui moved this task from Triage to In progress on the DBA board.May 13 2019, 7:58 PM
Marostegui added a subscriber: Marostegui.

Thanks for raising this.
Keep in mind that revision or logging are partitioned, as you pointed out, only on the special slaves (those on the recentchanges, watchlist etc groups). But to make it even more interesting, they are only partitioned on certain wikis (big ones normally). For instance it is partitioned on enwiki but not on small wikis like the ones on s3 as those are small enough not to run into problems. What is the baseline to decide which ones need it and which ones don't, is probably (and this is a guess, as I wasn't here when it was decided) query performance under a certain number of rows.
And if that wasn't enough, there are some wikis that have less partitions, for example revision on enwiki vs eswiki.

And of course, those special slaves also have different PKs for revision and ' logging` too.

I am not sure if we should partition something by log_actor but probably worth checking the query performance for the queries that will be arriving to those special slaves (which I guess it was the case for 085e3563ed at the time)

@Marostegui moved this task from In progress to Blocked external/Not db team on the DBA board.

I guess we should ask Performance then?

@Marostegui moved this task from In progress to Blocked external/Not db team on the DBA board.

I guess we should ask Performance then?

I moved it into that column of the DBA dashboard cause we have already given our opinion and we are blocked on a decision to see how/when we can move forward.