Page MenuHomePhabricator

[EPIC] Saved Groups in GrowthBook
Open, MediumPublic

Description

Saved Groups in GrowthBook lets us define groups once and reuse that combination elsewhere to target across experiments. We would use condition groups to have pre-set groups that our users can select.

Goal / scope

  • Produce a catalog of saved groups that teams need
  • Determine what's needed for v1 vs. later (i.e. we anticipate teams may want X in the future, but we don't need to do that today)
  • For each group, define the condition and the source for any metric-derived membership (e.g. top 10 wikis by pageviews -- what table/query/dashboard is that coming from?)
  • Establish naming protocol
  • Document an annual refresh plan

Product questions

  • What groups are our user teams looking for? (collect requests)
  • By what metrics? (e.g. Top 50 wikis by... pageviews? active editors? articles?)
  • How often do we revise these groups? IIUC, editing a saved group changes membership immediately, so we don't want to revise groups while a dependent experiment is running. We should time this carefully either around late Dec (before holiday deployment freeze) or early July (new FY)

Engineering questions

  • Would we deprecate groups that aren't being used? (do groups cost us anything?)
  • How might we test that this is working as intended?

Acceptance criteria

  • A catalog exists listing each group with: type, definition, computation source + cadence, owner.
  • Naming conventions are documented.
  • An revision plan including timing of changing groups is agreed upon.
  • v1 groups are created in GrowthBook

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

just an fyi - we removed Saved Groups functionality from the TK backend for now in https://gitlab.wikimedia.org/repos/data-engineering/test-kitchen/-/merge_requests/361 << some form of this will probably need to be reverted once we have a path forward to dealing with saved groups

fwiw - it was working to check against the GB API for the saved groups key in the experiment response - as we refine rules around this feature, we will need more precise logic for handling the different permutations around saved groups in conjunction with traffic targeting specific wikis

@KReid-WMF suggests, along the lines of versioning APIs instead of changing: "Top 10 wikis by pageviews for FY25/26" so that we can deprecate and also not disrupt anything in flight.

@phuedx mentions dblists as a similar phenomenon. But do dbnames serve us?