Your experiment schedule test failed validation and its status has been reverted to draft in GrowthBook.

Validation errors:

• variation ids must be changed from the default integers

While working on T429524: Experiments should always start at 14:30 UTC I have ended up writing some notes about how scheduled experiments work in GrowthBook and how our backend could deal with them. I think they could be useful for this task.

In fact, the approach I have chosen to implement T429524: Experiments should always start at 14:30 UTC could be pretty valid to cover what we need so that scheduled experiment work fine with GrowthBook + the poller

Per discussion in Slack w agreed on some points related to this work:

• Is this something which will change a lot as we move to custom hooks?

Good question! Scheduled experiments are technically a draft so most of the Custom Hooks validation we are implementing wouldn't apply for them until they effectively start. I didn't realize this before but I'm starting to think that we should tweak a bit the current implementation of Custom Hooks (pending review now) to validate a scheduled experiment once it's scheduled or approved like we are going to validate the regular experiments that are going to start. I guess that when someone schedules an experiment, they are not going to be there in front of the UI just in case there is some validation error. GrowthBook could show errors/warnings once the experiment is scheduled or that scheduling is approved

• Could the poller (or our custom hooks?) check at the scheduling stage and say “you’re good to go” before it actually switches on? That way, a user has the opportunity to adjust before it's supposed to go live, which reduces the risk that they'll have to wait another day for it to switch on.

It's doable. We can know that an experiment is a draft but it's scheduled or even that it's scheduled and approved. And we could do what I have mentioned in my message above

• How could we make more clear the actual time that it will start? (If before 14:30UTC, vs if after 14:30UTC, it'll actually be the next day after what's scheduled)

In this case I would say the poller should validate, at least, the drafts that represent a scheduled experiment, to adjust the start date once the experiment is scheduled or approved. That way users would be aware because that change will happen also in the GrowthBook UI, and before the experiment effectively started
In https://phabricator.wikimedia.org/T429524#12265206 I posted what I learned about how GrowthBook deals with scheduled experiments but I didn't realize it could be useful for Custom Hooks implementation and these cases you are mentioning here (edited)

I've written out the user stories and acceptance criteria in T438509: Implement scheduled/immediate experiment validation and deployment-window handling