[Customize GrowthBook] Customize GrowthBook UI
Closed, ResolvedPublic5 Estimated Story Points
Actions

Assigned To

Authored By

	Sfaci
	Apr 10 2026, 2:17 PM

Project Tags

Referenced Files

None

Subscribers

Aklapper

cjming

Sfaci

Description

Based on what we find via T422377: [Customize GrowthBook] Explore and evaluate what platform needs and what GrowthBook provides, we should customize the GrowthBook instances we have in both staging and production environments.

Regardless of the outcome for the spike above, there will be two different areas of work here:

Adding missing fields that are needed to register experiments to the GrowthBook UI. For example, a field to define a related Phabricator ticket or another one to set the Stream name that the corresponding piece of code will send events to. This first piece of work can be done simply by adding Custom Fields configured according to the specific needs for every specific field
New validation rules: Some additional validation rules to the ones that GrowthBook provides are needed. Custom Hooks is a very interesting feature for this purpose that is currently available for Features (but not for Experiments). We are with conversations with GrowthBook about having it also for Experiments. This would be the best way to address this piece of work, but they are others in case the first one cannot be used. So, the approach of the second one will depend on whether Custom Hooks are available or not for Experiments:
- Custom Hooks has been enabled for experiments: In this case a lot of additional validation rules can be added via that feature (probably all of them):
- Custom Hooks hasn't been enabled for experiments: In this case, the validation rules mentioned above will have to be implemented via Pre-launch checklists (which is more like a soft validation) + custom implementation of backend validation to ensure that experiments meet the requirements (in case a validation rule fails, experiment owners would be alerted via T427593: [GrowthBook Integration] [Spike] Add alerting and de-duplication for GrowthBook experiment validation failures)

Acceptance criteria

Additional fields needed in the GrowthBook UI
Missing fields have been added according to the approach defined in the related Spike

Additional validation rules in the case Custom Hooks have been enabled:
~~Custom Hooks have been defined for all GrowthBook UI validation rules defined in the related Spike~~ (not doable, Custom Hooks are not available yet for Experiments. We asked GrowthBook about that but we are still waiting for a response

Additional validation rules in the case Custom Hooks haven't been enabled:
A Pre-launch checklist has been defined with as many items as validation rules have been defined int he related Spike
Same validation rules have been implemented in the backend in the growthBookExperimentValidator component

Related Objects
Search...

Status	Assigned	Task
Resolved	Sfaci	T422376 [EPIC] Customize GrowthBook
Resolved	Sfaci	T422943 [Customize GrowthBook] Customize GrowthBook UI
Resolved	cjming	T428802 [Customize GrowthBook] Add TK backend validation for GrowthBook experiments
Resolved	Sfaci	T428805 [Customize GrowthBook] Populate prelaunch checklist per validation rules
Resolved	Sfaci	T428806 [Customize GrowthBook] Ensure fields in GrowthBook UI are accurately defined

Event Timeline

Sfaci created this task.Apr 10 2026, 2:17 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 10 2026, 2:17 PM

Sfaci added a parent task: T422376: [EPIC] Customize GrowthBook.Apr 10 2026, 2:17 PM

Sfaci updated the task description. (Show Details)May 19 2026, 11:24 PM

Adding reference here for Related OKRs and Owner/Team validation - https://gitlab.wikimedia.org/repos/data-engineering/test-kitchen/-/merge_requests/329#note_210225 discussion so we don't forget.

The main question is how to validate them as tags?

Example GB API response with single tags for each:

{
  "experiment": {
    "id": "exp_lf5cnmpdflnk3",
    "trackingKey": "sample-experiment",
    "name": "Sample Experiment",
    "type": "standard",
    "project": "prj_2Cb9wHcLGmgGaxzr9RdPyu",
    "hypothesis": "",
    "description": "Sample experiment for testing",
    "tags": [
      "Experiment Platform",
      "FY 25/26 SDS 2.3"
    ],
    "owner": "u_jtqkrpmmauur12",
    "dateCreated": "2026-05-20T02:17:13.348Z",
    "dateUpdated": "2026-05-27T03:47:30.239Z",
...

Example GB API response with multiple tags for each:

{
  "experiment": {
    "id": "exp_lf5cnmpdflnk3",
    "trackingKey": "sample-experiment",
    "name": "Sample Experiment",
    "type": "standard",
    "project": "prj_2Cb9wHcLGmgGaxzr9RdPyu",
    "hypothesis": "",
    "description": "Sample experiment for testing",
    "tags": [
      "Experiment Platform",
      "FY 25/26 SDS 2.3",
      "Editing",
      "FY25/26 WE 1.4",
      "FY25/26 WE 3.1",
      "Data Engineering"
    ],
    "owner": "u_jtqkrpmmauur12",
    "dateCreated": "2026-05-20T02:17:13.348Z",
    "dateUpdated": "2026-05-27T04:13:38.309Z",
    "archived": false,
...

I can spin this off into a subtask of this ticket if that would be helpful

In T422943#11966043, @cjming wrote:

Adding reference here for Related OKRs and Owner/Team validation - https://gitlab.wikimedia.org/repos/data-engineering/test-kitchen/-/merge_requests/329#note_210225 discussion so we don't forget.

The main question is how to validate them as tags?

I was thinking of something like what we have in Test Kitchen UI right now. We have somwhere all existing okrs and teams (in a database table and harcoded, respectively) so we could check if there is, at least, a tag for an OKR and also for a Team. We cannot validate whether they are the right ones, there is no way for now to validate that the experiment config has been tagged with the right team (as it's happening right now with TKUI) but we can check that, at least, one OKR and one Team are present in the experiment configuration. It's what TKUI is doing right now via a couple of MultiSelect fields for OKRs and related teams

The above is something that the backend could do pretty easily. Regarding the frontend side, we would depend on Custom Hooks. If we managed to get Custom Hooks for experiments, GrowthBook UI could probably check inline that an experiments has been tagged at least with one OKR and one team (Custom Hooks allows to make API request when extending the validation logic in the GB UI). At this time we have a local API endpoint for OKRs but, if needed, we could implement the corresponding one for Teams (and also the corresponding database table). It would be just a bit of additional work (same thing we have for OKRs but for teams).

I have updated the GrowthBook customization/validation document according to the above. I had forgotten to mention all this there.

Sfaci updated the task description. (Show Details)May 29 2026, 10:00 AM

Sfaci updated the task description. (Show Details)Jun 1 2026, 9:41 AM

Sfaci updated the task description. (Show Details)

KReid-WMF triaged this task as High priority.Jun 2 2026, 3:56 PM

KReid-WMF edited projects, added: Test Kitchen (Experiment Platform Sprint 25); removed: Test Kitchen.

KReid-WMF set the point value for this task to 5.

Sfaci claimed this task.Jun 9 2026, 11:59 AM

Sfaci moved this task from Ready for Development to In Progress on the Test Kitchen (Experiment Platform Sprint 25) board.

For now, because the related conversation with GrowthBook is not progressing, we will start this work considering that Custom Hooks won't be updated to support also Experiments. So, if nothing changes, Pre-Launch checklist + backend work will be the plan to add the needed validation rules

cjming mentioned this in T428802: [Customize GrowthBook] Add TK backend validation for GrowthBook experiments.Jun 10 2026, 7:33 PM

cjming added a subtask: T428802: [Customize GrowthBook] Add TK backend validation for GrowthBook experiments.Jun 10 2026, 7:35 PM

cjming mentioned this in T428805: [Customize GrowthBook] Populate prelaunch checklist per validation rules.Jun 10 2026, 7:40 PM

cjming added a subtask: T428805: [Customize GrowthBook] Populate prelaunch checklist per validation rules.

cjming mentioned this in T428806: [Customize GrowthBook] Ensure fields in GrowthBook UI are accurately defined.Jun 10 2026, 7:49 PM

cjming added a subtask: T428806: [Customize GrowthBook] Ensure fields in GrowthBook UI are accurately defined.

Sfaci removed Sfaci as the assignee of this task.Jun 10 2026, 10:08 PM

Sfaci moved this task from In Progress to Ready for Development on the Test Kitchen (Experiment Platform Sprint 25) board.

KReid-WMF closed subtask T428802: [Customize GrowthBook] Add TK backend validation for GrowthBook experiments as Resolved.Jun 22 2026, 3:33 PM

KReid-WMF closed subtask T428805: [Customize GrowthBook] Populate prelaunch checklist per validation rules as Resolved.Jun 25 2026, 3:08 PM

KReid-WMF closed subtask T428806: [Customize GrowthBook] Ensure fields in GrowthBook UI are accurately defined as Resolved.

KReid-WMF edited projects, added: Test Kitchen (Experiment Platform Sprint 26); removed: Test Kitchen (Experiment Platform Sprint 25).Jun 25 2026, 4:16 PM