Page MenuHomePhabricator

jsonschema-tools should add a 'latest' symlink
Closed, ResolvedPublic3 Estimated Story Points

Description

I'd like to be able to easily map from a stream config to a schema url. Even though each individual event is versioned with a specific schema, any given stream might contain events with different versions of the same schema. I want to add tooling that uses stream config to automatically map from a stream to the 'schema for the stream' to allow for easier integration with things like or Spark (and Flink) structured streaming.

To do this, I suggest we modify jsonschema-tools to maintain a 'latest' symlink to the latest materialized version of the jsonschema file in each schema directory.

Event Timeline

Milimetric moved this task from Incoming to Event Platform on the Analytics board.

Change 575372 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[schemas/event/secondary@master] Create latest symlinks

https://gerrit.wikimedia.org/r/575372

Change 575373 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[schemas/event/primary@master] Add latest symlinks

https://gerrit.wikimedia.org/r/575373

Change 575372 merged by Ottomata:
[schemas/event/secondary@master] Create latest symlinks

https://gerrit.wikimedia.org/r/575372

Change 575373 merged by Ottomata:
[schemas/event/primary@master] Add latest symlinks

https://gerrit.wikimedia.org/r/575373

Change 577311 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] schema.wikimedia.org - show 'latest' files as text in browser

https://gerrit.wikimedia.org/r/577311

Change 577311 merged by Ottomata:
[operations/puppet@production] schema.wikimedia.org - show 'latest' files as text in browser

https://gerrit.wikimedia.org/r/577311

Nuria set the point value for this task to 3.