Page MenuHomePhabricator

CI Support for Schema Registry
Open, NormalPublic

Event Timeline

Nuria triaged this task as Normal priority.Oct 11 2018, 10:03 PM
Nuria created this task.

CI for ensuring that schemas all have consistent meta field
CI for ensuring schema backwards compatibility
CI for schema linting, e.g. no camelCase, only snake_case, etc.

We actually have tests for all of these in the current event-schemas repo, but they're not perfect. What would I like to change:

CI for schema linting, e.g. no camelCase, only snake_case, etc.

This should become a special ESLint config or, more realistically, an ESLint plugin. There's no good plugin for ESLint YAML linting now, so we could either develop a solid general-purpose plugin and a set of rules for linting, or a specific schema linting plugin. There is though a JSON linting plugin, we might be able to convert YAML to JSON and have a set of rules for the JSON plugin.

CI for ensuring that schemas all have consistent meta field

We actually right now have a more powerful feature when we validate common properties by level. Every event must have a proper meta field. Every revision-related event must have revision-related fields etc. Right now it's achieved by copy-paste, but I would like to get rid of it. Given the full schema files will be generated by a pre-commit hook, we might want to utilize JSON-schema references and inheritance in the latest schema and expand them in the hook. It although raises questions on what to do when, for example, meta schema changes - do we create a new version of every schema? But backward compatibility and inclusion checks probably will require us to do so anyway.

CI for ensuring schema backwards compatibility

I have written a very naive schema backward compatibility checking code which I believe doesn't cover even half of the cases. There's no third-party library for checking JSON-schemas for compatibility (at least I couldn't find one) so I propose to actually write a proper module for that, publish it separately and use it here.

There is though a JSON linting plugin, we might be able to convert YAML to JSON and have a set of rules for the JSON plugin.

Perhaps the git hooks that generate the versioned files could render them as JSON instead of YAML anyway?

Ottomata moved this task from Backlog to Next Up on the EventBus board.Dec 5 2018, 10:04 PM
Ottomata removed Ottomata as the assignee of this task.Dec 5 2018, 10:09 PM
Nuria removed the point value for this task.Dec 5 2018, 10:16 PM
Ottomata assigned this task to Pchelolo.Jan 17 2019, 9:29 PM