Maniphest T207006

Set wb_changes_dispatch ROW_FORMAT=COMPRESSED on install and update
Closed, DeclinedPublic
Actions

Assigned To

None

Authored By

	Addshore
	Oct 15 2018, 9:45 AM

Description

T205865 revealed that the storage format for the wb_changes_dispatch was causing the selection of clients during the dispatching process to be super slow.
We should set the row format for the table to be compressed for install and update.

Link: https://mariadb.com/kb/en/library/xtradbinnodb-storage-formats/#compressed

Open questions:

Is this only a mariadb option? Is so, what can we do for mysql / do we need to?
In T205865#4665261 key block size of 8 is used, what should the standard wikibase install have?

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		Addshore	T205865 Investigate decrease in wikidata dispatch times due to eqiad -> codfw DC switch
		Declined		None	T207006 Set wb_changes_dispatch ROW_FORMAT=COMPRESSED on install and update

Event Timeline

Addshore triaged this task as Medium priority.Oct 15 2018, 9:45 AM

Addshore created this task.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 15 2018, 9:45 AM

Addshore moved this task from Inbox to Triaged High (100+) on the [DEPRECATED] wdwb-tech board.Oct 15 2018, 9:45 AM

Addshore moved this task from incoming to needs discussion or investigation on the Wikidata board.Oct 15 2018, 11:54 AM

Addshore mentioned this in T205865: Investigate decrease in wikidata dispatch times due to eqiad -> codfw DC switch.Oct 16 2018, 12:49 PM

Addshore added a project: Wikidata-Campsite.

Addshore moved this task from Incoming to Needs Work on the Wikidata-Campsite board.Oct 19 2018, 10:52 AM

Addshore added a project: DBA.Oct 31 2018, 4:53 PM

Addshore moved this task from Triage to Blocked external/Not db team on the DBA board.

loading a table with ROW_FORMAT=COMPRESSED fixed the issue but I am not sure compression is actually the cause, but maybe (I would actually bet on that) just a simple rebuilding from scratch after a many edits were suffered (something common after >1000M updates/inserts).

Compression is the way to go on Wikidata no doubt, but I am unsure I would set it up for all Wikibase installations... specially smaller ones. We can talk more in depth in the future. I would prefer honestly if we worked towards removing the table or at least separating it from the main metadata database (also thinking about Commons soon).

I would prefer honestly if we worked towards removing the table or at least separating it from the main metadata database (also thinking about Commons soon).

That is totally possible.
I guess it can say on the mysql servers? Where do you have in mind?
Perhaps we should discuss this in another ticket?

Addshore changed the task status from Open to Stalled.Jan 9 2019, 12:57 PM

Addshore removed a project: Wikidata-Campsite.

Addshore moved this task from needs discussion or investigation to in progress on the Wikidata board.Jun 21 2019, 11:31 PM

Addshore moved this task from in progress to needs discussion or investigation on the Wikidata board.Jun 22 2019, 10:32 PM