Implement the enrichment function
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	dcausse
	Sep 13 2022, 8:22 AM

Description

The indexable document have to be fetched from the CirrusSearch doc API using the flink AsyncIO operator.

AC:

The CirrusSearch doc API is called
Failures are retried 3 to 4 times
Missing revisions are identified and retried as long as the event is younger than 10sec (eventual consistency heuristic).
Error reasons are capture in a side-output

	Subject	Repo	Branch	Lines +/-
	Pass enriched revision-create to updater	search/cirrus-streaming-updater	master	+1 K -230

		Status	Subtype	Assigned	Task
		Open		None	T317045 [Epic] Re-architect the Search Update Pipeline
		Resolved		Gehel	T317611 Implement the enrichment function

pfischer changed the task status from Open to In Progress.Sep 14 2022, 7:12 AM

pfischer claimed this task.

pfischer triaged this task as Medium priority.

Change 832379 had a related patch set uploaded (by Peter Fischer; author: Peter Fischer):

[search/cirrus-streaming-updater@master] T317611 implement enrichment function

Change 842855 had a related patch set uploaded (by DCausse; author: Peter Fischer):

[search/cirrus-streaming-updater@master] Pass enriched revision-create to updater

Followup work (moving the schema out of the producer into the schema registry) will be done as part of T317202.

Change 842855 merged by jenkins-bot:

[search/cirrus-streaming-updater@master] Pass enriched revision-create to updater