[Task] Avoid unserializing metadata, especially diffs, when processing changes
Closed, InvalidPublic
Actions

Assigned To

None

Authored By

	daniel
	Aug 21 2015, 10:30 AM

Description

DiffChange objects tend to have a lot of data in the info blob. We should avoid:

loading the info blob from the database, if it is not needed
unserializing the info blob
unserializing the diff array in the info blob
instantiating Diff objects, if not needed

It seems like the only reason to look into the diff when dispatching changes is to find out whether the change added a sitelink. We could store that information separately, e.g. in a separate field in the info blob.

See T109088: [Task] Profiling and investigate how to improve performance of change dispatcher for the rationale.

NOTE: We may not need to unserialize change info at all, if we can remove the need of programmatic filtering. See in particular T111161: Subscribe client wikis immediately after adding sitelinks

NOTE: We may not need to put diffs into changes at all, see in particular T111356: [Story] wb_changes.change_info should only contain the minimum of information required for processing the change on the client.

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Invalid		None	T108944 [Epic] Improve change dispatching
		Invalid		None	T109821 [Task] Avoid unserializing metadata, especially diffs, when processing changes

Event Timeline

daniel created this task.Aug 21 2015, 10:30 AM

daniel raised the priority of this task from to Needs Triage.

daniel updated the task description. (Show Details)

daniel added projects: Wikidata, Performance Issue.

daniel subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 21 2015, 10:30 AM

daniel mentioned this in T109088: [Task] Profiling and investigate how to improve performance of change dispatcher.Aug 21 2015, 10:32 AM

daniel updated the task description. (Show Details)

daniel set Security to None.

Lydia_Pintscher added a parent task: T108944: [Epic] Improve change dispatching.Sep 1 2015, 11:49 AM

Lydia_Pintscher renamed this task from Avoid unserializing metadata, especially diffs, when processing changes to [Task] Avoid unserializing metadata, especially diffs, when processing changes.Sep 1 2015, 12:09 PM

Lydia_Pintscher triaged this task as High priority.

daniel updated the task description. (Show Details)Sep 2 2015, 10:15 AM

daniel updated the task description. (Show Details)Sep 3 2015, 2:20 PM

daniel updated the task description. (Show Details)

daniel mentioned this in T111356: [Story] wb_changes.change_info should only contain the minimum of information required for processing the change on the client..

JanZerebecki moved this task from incoming to needs discussion or investigation on the Wikidata board.Sep 10 2015, 10:40 PM