As WME Engineer I want to understand the Product Requirements and create a final schema for the release of wikidata on WME APIs.
Developer Notes:
As normal on data pipelines, each step should have it's own schema. Having multiple schemas will create a decoupling from producers and consumers. It will also allow us to focus on broader picture and future requirements rather than the immediate need. If we decouple eventstream listener for wikidata schema from the final schema we can introduce other consumers of that data in the future for better data collection and processing.
We suggest then to have 2 schemas, one for eventstream listener...that should focus on it's responsability and not be bound to the exact usage of wikidata.
TODO
- Gather product requirements, if not explicit in PRD and RFC.
- Define EventStream Listener Schema
- Define Wikidata Entity Schema
- Define Final Wikidata Schema - Ehi to create follow up tickets for schema, this is V1 (go with the SDK as is now)
- all comments in the wikidata article doc are resolved
- Present and Discuss with team
- Create the new avro and golang schemas
Acceptance Criteria
- Team agreement on schemas (product schema and event listener schema)
- Avro and Golang schemas merged to main in schema repository/submodule