We are mostly ready and able to create new schema repositories for uses other than production mediawiki events. However, while writing some documentation, I realized that we need to figure out how e.g. the analytics event schema repository will include the /common schema, currently defined in mediawiki/event-schemas repository.
Thus far, $refs in mediawiki/event-schemas are all confined to the local schema repository. When e.g. mediawiki/revision/create/current.yaml is materialized, the $ref's found are resolved relative to the configured schemaBaseUris, which defaults to the configured schemaBasePath, which is just the local working copy of the repository.
If we want a new 'analytics/event-schemas' (or whatever) repository to $ref: /common/1.0.0, we'll need a way for jsonschema-tools to resolve that $ref.
- Remote common: $ref: https://schemas.wikimedia.org/repositories/mediawiki/jsonschema/common/1.0.0. This works, but kinda sucks because it centralizes the common schema. The common schema MUST be resolvable over http when dependent schemas are materialized.
- Cached common: Create a /common/1.0.0 in analytics/event-schemas that has just $ref: https://schemas.wikimedia.org/repositories/mediawiki/jsonschema/common/1.0.0. This is slightly better in that other analytics/event-schemas schemas can then use a local (cached) $ref /common/1.0.0, but if any changes need to be made to the canonical common schema, it must be done in the remote repository, and then the local /common/1.0.0 must be materialized.
- git submodule: Use a git submodule to checkout a common schema repository and use it in local $refs. This might be nice if we can get the directory hierarchy to be consistent in all schema repositories, so the resolved $ref paths work the same in all of them.
- npm dependency: event schema repositories would use npm package.json to specify a dependency on common schema repositories they want to use. We could make postinstall we could make postinstall symlink the node_modules/wikimedia-common-schemas/jsonschema directory to something like ./repositories/wikimedia-common/jsonschema/, and then include that path in schemaBaseUris.
- copy/paste Just copy/paste /common schema into each schema repository. This might be the simplest solution. We'd still have a greatly reduced copy/paste within any given repository; current schema would only be duplicated once per repository. This might be confusing for our relative $ref URIs though: /common/1.0.0 in analytics/event-schemas would not be exactly the same as /common/1.0.0 in mediawiki/event-schemas. A pro AND con of this is that each schema repository could define their own common schema. Perhaps this would be more flexible anyway. We could still add some tests (perhaps via jsonschema-tools) to ensure that the minimal common fields exist ($schema, meta.dt, meta.id, meta.stream).