Background
https://phabricator.wikimedia.org/T422943#11998988 indicates we will not have Custom Hooks available for GrowthBook experiments, thereby necessitating backend validation during ingestion on the TK backend.
Canonical GrowthBook customization/validation spreadsheet captures validation rules.
Description
Per canonical spreadsheet tracking validation rules, add the following backend validation rules:
- Tracking Key (machine readable name)
- must be unique + varnish pattern
- GB experiment prevails when two experiments (GB and TKUI) have same machine-readable name
- Related OKRs (defined as tag in GB)
- Check if present via local API for OKRs
- We could use the local API we have for OKRs to check if at least one OKR if present as a tag
- Teams (defined as tag in GB)
- Check if present via local API for Teams
- Same thing as for OKRs. In this case we would need to build that local API + database table for teams
- Stream Name
- required custom field validated via Action API
- Traffic per Wiki
- check if present
- check here that 0 <= length(wikis) <= 100
- Risk Level
- check via current validation logic
- "https://wikitech.wikimedia.org/wiki/Test_Kitchen/Contextual_attributes#Privacy_considerations: If risk_level === 'Risk assessment pending' => experiment cannot run but there is no way to enforce that in GB Here we will need backend validation anyway"
- Security and Legal Review
- check via current validation logic
- Mandatory is risk_level is Tier 2: Medium or Tier 1: High
Acceptance Criteria
- Validation rules are implemented
- When any of the validation rules above fails, the experiment status is set to Draft as a way to "stop" it
- Relevant tests added/updated