In the parent task we are seeing we need a major bump for removing a regex pattern. Could we update the test suite to detect changes in pattern and accept loosening as backwards-compatible?
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T409207 Relax rules for identifiers in base product_metrics schemas | |||
| Declined | None | T411518 json schema tools: Can we allow changing the regex pattern for a schema field |
Event Timeline
Comment Actions
We could! I think technically changing the valid regex is backwards incompatible though, because it can potentially break consumers that expect values to match.
Other options:
- You can disable the compatibility check for a specific schema / version. Examples here: https://gitlab.wikimedia.org/repos/data-engineering/schemas-event-secondary/-/blob/master/.jsonschema-tools.yaml?ref_type=heads#L14
- You could just do a major schema version bump. While we recommend versioning streams (and tables!) by major version, most streams don't do this. There is nothing (other than schema compatibility CI) that checks the compatibility of schema versions. You can bump the major version, start producing events of the new major version to the same old stream, and they will be valid and will likely not break anything (since Hive is the only real consumer which doesn't care about the values of the strings).
Comment Actions
no immediate need for this - please reopen otherwise (ideal would be to reduce friction on major version bumps for schemas)