Description

As part of T211248: Modern Event Platform: Stream Intake Service: Migrate eventlogging-service-eventbus events to eventgate-main, we migrated revision-create to a slightly modified format We no longer user meta.topic, but instead, use a more generic meta.stream. This broke the change-prop requests to ORES /precache with ORES returning a 400 error: "Must provide the 'topic' parameter".

Petr has deployed a temporary workaround in change-prop to keep things working.

ORES should do one of the following

not use meta.topic (is it needed there for /precache?)
switch to using meta.stream
- and/or make the stream field configurable, e.g. stream_field and set it to meta.stream.

Once this is done Petr can remove his temporary workaround.

Related Objects
Search...

Status	Assigned	Task
Resolved	Ottomata	T185233 Modern Event Platform
Resolved	Ottomata	T201068 Modern Event Platform: Stream Intake Service
Resolved	Ottomata	T211248 Modern Event Platform: Stream Intake Service: Migrate eventlogging-service-eventbus events to eventgate-main
Resolved	• Pchelolo	T228688 Fix revision-score event production in change-prop after migration of revision-create to eventgate-main
Resolved	• ACraze	T229848 ORES deployment, Early August 2019
Resolved	Halfak	T228689 Fix "Must provide the 'topic' parameter" in ORES /precache endpoint

Event Timeline

Ottomata created this task.Jul 22 2019, 7:20 PM

Ladsgroup unsubscribed.Jul 22 2019, 10:26 PM

Is there documentation for meta.stream? Should we expect the same data in a different field?

I'm not sure about the doc, @Ottomata might know more, but yes, the only change is .meta.topic renamed to .meta.stream, the value would be the same.

https://github.com/wikimedia/eventgate#streams

And we set stream_field to meta.stream in https://gerrit.wikimedia.org/r/plugins/gitiles/eventgate-wikimedia/+/refs/heads/master/config.yaml#97 (and also helm values files for deployment)

It looks like we are using "meta.topic" here: https://github.com/wikimedia/ores/blob/master/ores/wsgi/util.py#L186

The only difference I see is that meta.topic included the datacenter and meta.schema is just a raw schema name (which is way easier to work with). If that's right, then this PR should be good: https://github.com/wikimedia/ores/pull/328

Halfak edited projects, added Machine-Learning-Team (Active Tasks); removed Machine-Learning-Team.Jul 23 2019, 5:49 PM

Halfak moved this task from Parked to Review on the Machine-Learning-Team (Active Tasks) board.

Great! I left a comment there. It is 'meta.stream' not meta. schema. The [[ https://github.com/wikimedia/eventgate#eventvalidator-class--event-schema-urls | schema uri now lives at $schema top level. ]]