Implement buffering in DispatchingTermLookup
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	WMDE-leszek
	Nov 1 2016, 1:47 PM

Description

Following up T149583 BufferingTermLookup(s) used in client should work with foreign entity ids.
We need to figure out how to handle buffering with multiple repositories.

Details

	Subject	Repo	Branch	Lines +/-
	Add DispatchingTermBuffer.	mediawiki/extensions/Wikibase	master	+347 -0

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
Declined		dchen	T118706 Conduct heuristic evaluation of image upload and insert flow in VisualEditor
Open		None	T115858 Design improvements for mw.ForeignStructuredUpload.BookletLayout
Open		None	T115865 Insert image in content immediately after it's uploaded, skipping the "General settings" step
Duplicate		None	T115864 Figure out if the description of the image can be used as the caption on-wiki
Open	Feature	None	T53032 When inserting an image, set its caption by default to be the Commons image description
Open	Feature	None	T39534 Wikimedia Commons should support searching by color
Duplicate		None	T39535 Wikimedia Commons should support filtering by color
Resolved		None	T19503 Provide metadata support on Wikimedia Commons
Resolved		None	T51662 VisualEditor: Use Multimedia/Wikidata's proposed rich structured meta-data in the image insertion dialog
Resolved		None	T68108 [Epic] Store media information for files on Wikimedia Commons as structured data
Open		None	T109579 [Epic] Give more sister projects access to Wikidata
Open		None	T187900 There is no way to reference a specific quote on Wikiquote
Stalled		None	T71753 [Story] Wikibase / Wikidata support on Wikiquote
Resolved		Lydia_Pintscher	T146637 Wikidata 2016 Q4 goals
Resolved		Lydia_Pintscher	T150179 Wikidata 2017 Q1 goals
Resolved		Lydia_Pintscher	T76007 [Epic] add ability to link/refer to foreign items and properties (federation)
Resolved		Lydia_Pintscher	T149580 Allow access to entity data stored on "foreign" repositories (federation)
Resolved		WMDE-leszek	T155623 Allow optional use of data retrieval services from Client in Repo
Resolved		WMDE-leszek	T148141 Use dispatching services in Client
Resolved		Jakob_WMDE	T149686 Implement buffering in DispatchingTermLookup
Resolved		WMDE-leszek	T155633 Make it possible to use DispatchingServiceFactory as a watcher for entity change events

Event Timeline

WMDE-leszek created this task.Nov 1 2016, 1:47 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 1 2016, 1:47 PM

We should probably use this opportunity to unwind how caching works inside of the various term related services.

You might want to make BufferingTermLookup a general wrapper around another EntityTermLookupBase (instead of it using a TermIndex directly).

Please keep in mind that the defining feature of BufferingTermIndex is prefetching, not caching. It's related, but different, concern.

daniel added a parent task: T149580: Allow access to entity data stored on "foreign" repositories (federation).Nov 1 2016, 2:13 PM

daniel added a project: Wikidata.

In T149686#2759776, @daniel wrote:

Please keep in mind that the defining feature of BufferingTermIndex is prefetching, not caching. It's related, but different, concern.

True… the name doesn't really reflect that, though. Often it's enough to have a caching service that allows batch lookups to implement pre-fetching, I guess that is also true here.

daniel added a parent task: T148141: Use dispatching services in Client.Nov 1 2016, 5:47 PM

Please see me implementation notes in T148141#2760606

Jakob_WMDE claimed this task.Nov 3 2016, 9:52 AM

Jakob_WMDE moved this task from accepted to doing on the WMDE-TLA-Team board.

Change 320225 had a related patch set uploaded (by Jakob):
WIP: Add DispatchingTermBuffer.

https://gerrit.wikimedia.org/r/320225

gerritbot added a project: Patch-For-Review.Nov 7 2016, 4:55 PM

WMDE-leszek mentioned this in T149583: Create a dispatching TermLookup, to work with foreign entities.Nov 8 2016, 9:04 AM

WMDE-leszek moved this task from doing to needs review on the WMDE-TLA-Team board.Nov 8 2016, 12:24 PM

Lydia_Pintscher moved this task from incoming to in progress on the Wikidata board.Nov 10 2016, 1:48 PM

Change 320225 merged by jenkins-bot:
Add DispatchingTermBuffer.

https://gerrit.wikimedia.org/r/320225