Page MenuHomePhabricator

xLab: Add validation to the Duration field
Closed, ResolvedPublic5 Estimated Story Points

Assigned To
Authored By
Sarai-WMF
Apr 29 2025, 11:48 AM
Referenced Files
F62557758: date_backspace.webm
Jun 27 2025, 2:09 AM
F62400598: Screenshot 2025-06-20 at 10.31.27.png
Jun 20 2025, 8:38 AM
F62400585: Screenshot 2025-06-20 at 10.29.34.png
Jun 20 2025, 8:38 AM
F62363216: Jun-17-2025 13-57-29.gif
Jun 17 2025, 12:49 PM
F62362316: Screenshot 2025-06-17 at 12.15.01.png
Jun 17 2025, 12:49 PM
F62349049: Screenshot 2025-06-16 at 12.30.54.png
Jun 16 2025, 10:31 AM
F62275229: Screenshot 2025-06-09 at 19.11.07.png
Jun 9 2025, 5:28 PM
F62274644: Screenshot 2025-06-09 at 17.54.51.png
Jun 9 2025, 3:56 PM

Description

Problems
  1. Lack of client-side validation: xLab currently doesn't validate users' input to the Start or End date fields, which means that artifacts with invalid durations can be saved (e.g. set to start in the past; or where the end date is set before the start date, etc.).
  1. It's allowed to activate experiments immediately: xLab users can currently enable experiments right after configuring them. This means that their activation could technically happen before Varnish nodes have a copy of the experiment configuration. This would impede experiment enrollment and prevent the successful collection of data.
Suggested solution
Improve the validation of the Duration field

1. Implement client-side validation: The following errors should be displayed under the last modified fields on blur. The errors should persist on submit (see 2) and be listed after one another in the edge case they are concurrent:

Start date must be before end dateStart date must be 24h in the future (Experiment form) *Start date must be in the future (Instrument form) *
Screenshot 2025-06-04 at 14.52.20.png (1,036×295 px, 35 KB)
Screenshot 2025-06-04 at 15.05.50.png (1,034×332 px, 38 KB)
Screenshot 2025-06-09 at 17.54.51.png (1,428×404 px, 52 KB)

(*) Please note that the 24h ahead rule only applies to experiments

End date must be after start dateEnd date must be in the future
Screenshot 2025-06-04 at 14.53.15.png (1,096×313 px, 38 KB)
Screenshot 2025-06-04 at 15.07.35.png (1,121×318 px, 38 KB)

1.1. Cross-field validation: Please note that for the cross-field validation errors "Start date must be before end date" and "End date must be after start date", the client-side error message will also be displayed on blur only under the last field that was edited (where the relational date validation error was made). Only if the error isn't corrected before submitting will both fields display messages at the same time:

On blur on last edited fieldOn submit
Screenshot 2025-06-04 at 15.44.27.png (1,008×282 px, 34 KB)
Screenshot 2025-06-04 at 15.39.08.png (1,145×329 px, 43 KB)

1.2. Concurrent errors: Also note that, in the edge case in which client-side errors overlap, they'll be displayed as a list under the affected field(s)

On blur on last edited fieldOn submit
Screenshot 2025-06-04 at 15.43.17.png (971×310 px, 37 KB)
Screenshot 2025-06-04 at 15.14.59.png (1,107×395 px, 53 KB)

2. Implement server side validation: If client-side errors were present at the time users save their configurations (see 1.1), said errors will persist on submit. We'll also display the required field error messages inline, following the new validation improvements (see parent task, T372952):

Screenshot 2025-06-04 at 15.10.28.png (1,094×314 px, 37 KB)

Implement a 24h buffer interval before saving and activating experiments

To ensure that Varnish nodes have a copy of experiment configurations with enough time between their activation and start dates, we should ensure that users save and activate their experiments at least 24 hours prior to the defined start date.

1. Client-side validation will be triggered on blur if users input a start date less than 24h in the future (Please see Open question #1)2. On the Experiments and Instruments table, activation will be disabled for experiments set to start in less than 24 hours. Users will find an informative description under a disabled “Turn on” menu option:3. Copy updates: (1) The Duration field's description is updated to indicate the system's fixed time; (2) In the experiment form, a new helper text is included under the start date for error prevention:
Screenshot 2025-06-04 at 15.05.50.png (1,034×332 px, 38 KB)
Screenshot 2025-06-09 at 19.11.07.png (1,353×776 px, 137 KB)
Screenshot 2025-06-04 at 16.05.16.png (1,277×340 px, 52 KB)
Open questions

Validating that the start date is 24h ahead in the future on the client side implies considering the current user's time at the moment of input. The feasibility of this is unknown. If not possible, we might just validate that start date is in the future on the client side, and validate whether there's a 24h buffer at the time of creation only on submit.

Acceptance criteria
  • Users’ input to the Start and End date field in the Duration module are validated in the client and server side to ensure that dates are correct
  • Users can only save and activate experiments at least 24 hours prior to their set start date

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

The final launch time for experiments and instruments is in the process of being determined as part of T395025: xLab: Add fixed experiment launch window to deployments calendar. If modified from 00:00am UTC, then the following copy elements will need to be adjusted:

  • The Duration field's description
  • The informative description included under the disabled “Turn on” menu option in the Experiments and instruments table

(See ponts 2 and 3 of the section "Implement a 24h buffer interval before saving and activating experiments" in this task's description)

Milimetric triaged this task as Medium priority.Jun 12 2025, 3:31 PM
Milimetric set the point value for this task to 5.
Sfaci updated the task description. (Show Details)

The MR above is ready for review.

Regarding the open question above (in the description)

Open questions
Validating that the start date is 24h ahead in the future on the client side implies considering the current user's time at the moment of input. The feasibility of this is unknown. If not possible, we might just validate that start date is in the future on the client side, and validate whether there's a 24h buffer at the time of creation only on submit.

The current implementation is validating that the start date is set 24h before the current time in the client side (as UTC) and also same validation is run in the server side according time there (as UTC as well) when the form is submitted

Also, according to T395025: xLab: Add fixed experiment launch window to deployments calendar where a deployment window for 14:30h has been validated, the corresponding message in the UI has been updated to talk about that time.

Sfaci updated Other Assignee, added: Sarai-WMF.
Sfaci subscribed.

Change #1159526 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/deployment-charts@master] xLab: Deploy v0.7.0 release to staging

https://gerrit.wikimedia.org/r/1159526

Change #1159526 merged by jenkins-bot:

[operations/deployment-charts@master] xLab: Deploy v0.7.0 release to staging

https://gerrit.wikimedia.org/r/1159526

This works really smoothly! Thank you, @Sfaci 💯

I only found what seems to be an issue:

  1. Start date must be 24h in the future (Experiment form): Entering the date 18/06/2025 on 17/06/2025 at UTC 10:15 triggered the "Start date must be at least 24h in the future" error. The error only went away when I inputted 19/06/2025.
    Screenshot 2025-06-17 at 12.15.01.png (670×239 px, 31 KB)

And an improvement opportunity:

  1. Cross-field validation ("Start date must be before end date" and "End date must be after start date"): Right now, when the "before" and "after" errors are corrected by updating one of the fields, the parallel field still displays the error even if it no longer applies, which can be confusing. The error only finally disappears when the latter is focused and blurred (even if an update of the value wasn't necessary anymore):
    Jun-17-2025 13-57-29.gif (770×276 px, 61 KB)
    I was wondering if it'd be possible to remove cross-field errors when they are corrected by modifying just one of the fields and blurring. Please note that the current cross-field validation doesn't follow the original specification (to display both messages if errors persist on submit), but I believe that the implemented approach (displaying said errors on blur) works well, and it would really improve if we could apply this last fix.

Thanks @Sarai-WMF, good catches! The above MR should fix the issue and add the improvement:

Start date must be 24h in the future (Experiment form): Entering the date 18/06/2025 on 17/06/2025 at UTC 10:15 triggered the "Start date must be at least 24h in the future" error. The error only went away when I inputted 19/06/2025.

This is now fixed

Cross-field validation

Good idea! I have added a cross-date validation so both dates, start and end, are validated every time the blur effect happens for any of them

Once we deploy a new release with these changes to staging, we will ping you to test it

Change #1161050 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/deployment-charts@master] xLab: Deploy v0.7.1 release to staging

https://gerrit.wikimedia.org/r/1161050

Change #1161052 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/deployment-charts@master] xLab: Deploy v0.7.1 release to production

https://gerrit.wikimedia.org/r/1161052

Change #1161050 merged by jenkins-bot:

[operations/deployment-charts@master] xLab: Deploy v0.7.1 release to staging

https://gerrit.wikimedia.org/r/1161050

Change #1161052 merged by jenkins-bot:

[operations/deployment-charts@master] xLab: Deploy v0.7.1 release to production

https://gerrit.wikimedia.org/r/1161052

Start date must be 24h in the future (Experiment form): Entering the date 18/06/2025 on 17/06/2025 at UTC 10:15 triggered the "Start date must be at least 24h in the future" error. The error only went away when I inputted 19/06/2025.

Cross-field validation ("Start date must be before end date" and "End date must be after start date"): Right now, when the "before" and "after" errors are corrected by updating one of the fields, the parallel field still displays the error even if it no longer applies, which can be confusing. The error only finally disappears when the latter is focused and blurred (even if an update of the value wasn't necessary anymore):

Hi @Sarai-WMF, There is already a new release deployed to production where the last issue and the new improvement have been implemented. It's ready for you to test.
Thanks!

Looking great, @Sfaci! Thanks for all the changes 🙏🏻

There's just one last thing:

  • In the latest version of the implementation, if the value of the Start date field on the experiment form is in the past, the field displays two error messages (the error intended for A/B tests and the one intended for instruments):
Issue in the A/B test formInstrument form
Screenshot 2025-06-20 at 10.29.34.png (1,650×682 px, 121 KB)
Screenshot 2025-06-20 at 10.31.27.png (1,658×452 px, 73 KB)

Hopefully that's it! Adding a note for the record, that the final implementation doesn't match the behavior defined in the specs for cross-field validation (point 1.1): parallel errors are displayed at the same time under both fields, but we agreed that this is acceptable 👍🏻

Sfaci updated Other Assignee, removed: Sarai-WMF.
Sfaci added a subscriber: EChukwukere-WMF.

There's just one last thing:
In the latest version of the implementation, if the value of the Start date field on the experiment form is in the past, the field displays two error messages (the error intended for A/B tests and the one intended for instruments):

This MR is ready for review and aims to fix that pending issue

Pending issues in T372952: (stretch) xLab: Add better validation to forms and T397510: xLab: Remove persistent required errors are also included in this MR

Change #1162089 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/deployment-charts@master] xLab: Deploy v0.7.2 release to staging

https://gerrit.wikimedia.org/r/1162089

Change #1162089 merged by jenkins-bot:

[operations/deployment-charts@master] xLab: Deploy v0.7.2 release to staging

https://gerrit.wikimedia.org/r/1162089

Sfaci updated Other Assignee, added: EChukwukere-WMF.

@Sarai-WMF @EChukwukere-WMF The last pending issue is already fixed and deployed on staging for you to review:

There's just one last thing:
In the latest version of the implementation, if the value of the Start date field on the experiment form is in the past, the field displays two error messages (the error intended for A/B tests and the one intended for instruments):

Details here

Looking great! Thanks for the last fix, @Sfaci 💯

I noticed the smallest copy issue. I'm not sure whether we should fix the following as part of this task, but currently the time indicated in the Duration field's description – 14:30pm – is redundant. The description should just say "14:30 UTC".

Sfaci updated Other Assignee, removed: EChukwukere-WMF.

Test status: QA FAIL

Select the start and End dates such that the dates display no validations. In any of the dates ( start or end) select the day, month or year and then hit the backspace button.

This clears that date unit entry. This field does not throw any validation and prevents users from creating the experiment.

This should at least throw a validation message

I noticed the smallest copy issue. I'm not sure whether we should fix the following as part of this task, but currently the time indicated in the Duration field's description – 14:30pm – is redundant. The description should just say "14:30 UTC".

Thanks @Sarai-WMF!!. We have already prepared a MR to fix it.

Select the start and End dates such that the dates display no validations. In any of the dates ( start or end) select the day, month or year and then hit the backspace button.
This clears that date unit entry. This field does not throw any validation and prevents users from creating the experiment.
This should at least throw a validation message

Good catch @EChukwukere-WMF! This is something we already discussed at some point, we decided to remove input-type from some other fields where we were using it and experiencing same issue (Traffic, email and phabricator task). Our validation process is covering that part and we could remove them. But we need that attribute for dates for the button and the popup calendar to appear and we didn't do anything there. I guess we should have filed a ticket to keep an eye on this because we have ended up forgetting the thing until you have found the issue again.

Apparently the built-in form validation is preventing the form from being validated via vestjs (because the input is invalid) and the validation suite we have implemented doesn't have the opportunity to do its job. I think there are some ways to bypass this built-in validation or maybe we can merge it with ours and fix the issue. If you are ok, I'll create a new ticket to explore an implement a fix for this issue.

@EChukwukere-WMF I have filed this ticket to try to fix the issue you mentioned above: T398063: Make vestjs validation work with HTML built-in validation

I'll move this ticket to Sign-Off to mean that we still need to deploy to production. Please, let me know if that works for you

Thanks @Sfaci , we can now move this ticket to done.

Change #1165181 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/deployment-charts@master] xLab: Deploy v0.7.5 release to staging

https://gerrit.wikimedia.org/r/1165181

Change #1165181 merged by jenkins-bot:

[operations/deployment-charts@master] xLab: Deploy v0.7.5 release to staging

https://gerrit.wikimedia.org/r/1165181

Change #1165182 had a related patch set uploaded (by Clare Ming; author: Clare Ming):

[operations/deployment-charts@master] xLab: Deploy v0.7.5 release to production

https://gerrit.wikimedia.org/r/1165182

Change #1165182 merged by jenkins-bot:

[operations/deployment-charts@master] xLab: Deploy v0.7.5 release to production

https://gerrit.wikimedia.org/r/1165182