Page MenuHomePhabricator

prediction_classification events rejected by EventGate due to extra schema properties
Closed, ResolvedPublicBUG REPORT

Description

Steps to replicate the issue (include links if applicable):

  • Deploy a service that consumes page change events and publishes predictions classification events (e.g. revertrisk-multilingual, revertrisk-language-agnostic) with EventGate publishing enabled
  • Submit a page_change event with additional fields
  • Observe the EventGate response or errored schema logs

What happens?:
EventGate rejects the event with schema validation errors:

  • .page should NOT have additional properties
  • .performer should NOT have additional properties
  • .prior_state.revision.editor should NOT have additional properties
  • .revision.editor should NOT have additional properties

What should have happened instead?:

The event should not be rejected by EventGate. The constructor should only include properties defined in prediction_classification_change/1.2.0, stripping any extra fields that leaked from newer page_change fragment schemas.
In events.py all the event properties are copied which eventually ends up in adding them to the output. We should explicitly construct the event dict, picking only the properties defined in the target schema

Services affected (which need to be deployed after this change):

  • revertrisk-language-agnostic
  • revertrisk-multilingual
  • outlink-topic-model
  • article-country

Event Timeline

Change #1191305 had a related patch set uploaded (by AikoChou; author: AikoChou):

[machinelearning/liftwing/inference-services@main] events: construct new prediction classification event independently

https://gerrit.wikimedia.org/r/1191305

Change #1191305 merged by jenkins-bot:

[machinelearning/liftwing/inference-services@main] events: construct new prediction classification event independently

https://gerrit.wikimedia.org/r/1191305

Change #1290699 had a related patch set uploaded (by AikoChou; author: AikoChou):

[operations/deployment-charts@master] ml-services: update article-country, revertrisk, outlink, revertrisk-multilingual image tags

https://gerrit.wikimedia.org/r/1290699

Change #1290699 merged by jenkins-bot:

[operations/deployment-charts@master] ml-services: update article-country, revertrisk, outlink, revertrisk-multilingual image tags

https://gerrit.wikimedia.org/r/1290699

The fix has been deployed to all the services affected. Fixed in https://gerrit.wikimedia.org/r/1191305 by explicitly constructing the prediction_classification_change event field by field using helper functions that only pick fields defined in prediction_classification_change/1.2.0, so upstream schema changes can no longer leak into emitted events.

As a future improvement, we could adopt a schema-driven stripping approach (e.g. extending the jsonschema validator) to automatically handle schema bumps without needing to update the helpers manually.