The streaming updater consumer should stop accumulating patches if it cannot handle them
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	dcausse
	Oct 29 2020, 9:06 AM

Description

The KafkaStreamConsumer should stop accumulation of patches if it is going to fail.

java.lang.IllegalArgumentException: Cannot accumulate data for entity: Q16994300
        at org.wikidata.query.rdf.updater.consumer.PatchAccumulator.accumulate(PatchAccumulator.java:104)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
        at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
        at org.wikidata.query.rdf.updater.consumer.KafkaStreamConsumer.poll(KafkaStreamConsumer.java:131)
        at org.wikidata.query.rdf.updater.consumer.StreamingUpdaterConsumer.lambda$run$0(StreamingUpdaterConsumer.java:46)
        at org.wikidata.query.rdf.common.TimerCounter.time(TimerCounter.java:51)
        at org.wikidata.query.rdf.updater.consumer.StreamingUpdaterConsumer.run(StreamingUpdaterConsumer.java:46)
        at org.wikidata.query.rdf.updater.consumer.StreamingUpdate.main(StreamingUpdate.java:49)

Details

	Subject	Repo	Branch	Lines +/-
	Respect PatchAccumulator.canAccumulate()	wikidata/query/rdf	master	+67 -18

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		Gehel	T244590 [Epic] Rework the WDQS updater as an event driven application
		Resolved		dcausse	T266750 The streaming updater consumer should stop accumulating patches if it cannot handle them

Event Timeline

dcausse created this task.Oct 29 2020, 9:06 AM

Restricted Application added a project: Wikidata. · View Herald TranscriptOct 29 2020, 9:06 AM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

dcausse claimed this task.Oct 29 2020, 2:47 PM

dcausse added a project: Discovery-Search (Current work).

dcausse moved this task from Incoming to Needs review on the Discovery-Search (Current work) board.

Change 637505 had a related patch set uploaded (by DCausse; owner: DCausse):
[wikidata/query/rdf@master] Respect PatchAccumulator.canAccumulate()

https://gerrit.wikimedia.org/r/637505

gerritbot added a project: Patch-For-Review.Oct 29 2020, 2:48 PM

dcausse renamed this task from The streaming updater consumer should stop accumulating patches if it cannot handle to The streaming updater consumer should stop accumulating patches if it cannot handle them.Oct 29 2020, 3:14 PM

Gehel added a parent task: T244590: [Epic] Rework the WDQS updater as an event driven application.Oct 30 2020, 1:49 PM

CBogen moved this task from Incoming to Current work on the Wikidata-Query-Service board.Nov 2 2020, 3:06 PM

Change 637505 merged by jenkins-bot:
[wikidata/query/rdf@master] Respect PatchAccumulator.canAccumulate()

https://gerrit.wikimedia.org/r/637505

dcausse moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.Nov 16 2020, 6:08 PM

Maintenance_bot removed a project: Patch-For-Review.Nov 16 2020, 6:10 PM

Gehel closed this task as Resolved.Nov 23 2020, 1:23 PM