Page MenuHomePhabricator

Modern Event Platform: Schema Registry Implementation
Closed, DuplicatePublic0 Estimated Story Points


Ticket proliferation disambiguration!

The canonical storage of event schemas will be git. We already have the mediawiki/event-schemas repository that we can use as a basis for this work. There will likely also be other schema repositories (e.g. analytics, etc.) that will be separate from mediawiki/event-schemas for production use.

This ticket will be used to track tasks and implementation design.

On 2018-10-12, @Pchelolo and @Ottomata brainstormed implementation ideas. Much of the implementation work to be done is around CI and development workflows. Some of this is already done for mediawiki/event-schemas, but we need to do more. I'll try and collect some of the things we need to implement.

  • editing of schemas should be done to the current schema version.
  • JSON $ref pointers can be used only in the current schema version.
  • $ref pointers to other schemas must be strongly versioned. E.g. if we factor out the meta schema,
  • every event that uses it will point to a specific version of meta, e.g. meta/3, or meta/4.
    • versioned $ref pointers in schemas must be manually upgraded by editing the schema and creating a new schema version.
  • This will ensure that any changes to referenced schemas will not affect user schemas until they manually update the referenced version. (This is how dependencies normally work anyway.)
  • git hooks will dereference current to generate standalone explicitly committed versioned schema files.
  • next schema version number can be computed from upstream branch
    • e.g. if upstream origin/master has revision/create/3 as the latest, a change to revision/create/current will generate revision/create/4 for review. If local checkout of master has revision/create/4, but upstream origin/master still only has revision/create/4, a change to revision/create/current will regenerate revision/create/4.
  • if only a code comment or description field change in current schema, don't generate a new schema version.
  • backwards compatibility library T206889 ensure changes are backwards compatible in git hook and also CI.