Page MenuHomePhabricator

Metrics Platform Schema testing & feedback: Morten
Closed, ResolvedPublic

Description

Context:

As we work on the Metrics Platform, trying to standardize how we ingest and capture common fields as well as consistently handle bespoke/custom fields, we will be introducing a new schema format. While designing the schema for bespoke/custom fields, we realize that if we want to support the ability to add and remove fields at-will, it may result in more effort to parse or query the data directly. Thus the Metrics Platform team would like to work with the Product Analytics team, to define a balance between the level of flexibility desired vs the ease-of-use when it comes to writing queries.

What questions are we trying to answer?

  • Does querying become more difficult the more flexibly we handle bespoke/custom fields?
  • Does flexibility of adding new fields take precedence over ease-of-use when querying?
  • What is the balance we need to strike between flexibility and querying? "

How long will this take?
This should not take longer than 20-30 minutes.

What are we testing?
Please refer to 2021-08-05 Team Sharing notes.

Details

Due Date
Aug 21 2021, 4:00 AM

Event Timeline

mpopov moved this task from Next 2 weeks to Blocked on the Product-Analytics (Kanban) board.

Metrics Platform has requested this testing & feedback to be done in the next 2 weeks so I moved it to Kanbab, but the work here is currently blocked on availability of the 2nd dataset (additional work by Metrics Platform).

mpopov set Due Date to Aug 21 2021, 4:00 AM.
nettrom_WMF added a subscriber: Mayakp.wiki.

@mpopov & @Mayakp.wiki : I left some comments about stumbling blocks (e.g. missing data) in my feedback that you might want to consider fixing before other team members start digging into this. I'll close this task as resolved, feel free to ping me on Slack with questions about anything if follow up is needed.