Problem
xLab currently provides a “Project and sample size” field that allows defining the size of the user sample to be enrolled in each one of the wikis involved in their experiment. This field presents some of issues:
- Selecting “projects” is inaccurate, since enrollment takes place on the specific domains of individual wikis (e.g., the wiki available in en.wikipedia.org wiki instead of Wikipedia as a project)
- Project selection is based on dbnames, which isn't completely user-friendly (e.g., "eswiki" instead of "Spanish Wikipedia");
- The notion of "Sample size" is an improvement from the previous "Sample rate" (see MPIC Usability Evaluation Findings), but we could make the concept even clearer if we used terminology and input values that are more accessible for users
Suggested solution
We should improve the “Project and sample size” module, applying the following adjustments:
- We replace the notion of ‘project’ by ‘wiki’ everywhere in the module (see Copy adjustments)
- Menu options display full, user-friendly wiki names instead of dbnames
- ‘Sample size’ is replaced by ‘Traffic’, which value is now inputted as a percentage from 1 to 100.
- Moreover, the update of the “Project and sample size” field to ”User traffic per wiki” justifies amending the title of the current “Sampling” section to, instead, “Traffic and variations”.
Overview of changes:
| Copy adjustments | Menu adjustments | Validation |
 |  |  |
| Labels, placeholders and descriptions should be adjusted to reflect the switches to the notions of ‘wiki’ and ‘traffic’. | 1. To facilitate locating the right option, the wiki selection menu will only display a max. of 7 suggestions by default. The existence of further options is indicated by the scroll bar displayed by the menu. 2. The menu displays full, user-friendly wiki names. | Validation scenarios: 1) The Traffic field will be validated on the client side if users enter a value greater than 100; 2) The Traffic field will be validated on submit if users leave it empty when a corresponding wiki has been selected; and 3) The Wiki field will be validated on submit if left empty when the corresponding traffic has been selected. |
Find more details in the Figma specifications
Notes
- The description of the new ‘User traffic per wiki’ module includes a link to a documentation page that doesn’t exist yet. We decided to leave it in the designs and specs as a reminder of the relevance of providing more information and directions to users while filling in this field ( e.g. regarding the recommended amount of traffic they should select per wiki).
- The overall 'User traffic per wiki' field validation will be implemented as part of T372952: (stretch) xLab: Add better validation to forms. We'll just have to adjust some of the logic (e.g. due to the usage of percentages) as well as the copy of the error messages as part of this ticket.
Acceptance criteria
- The ‘Project and sample size’ field in xLab’s experiment and instrument forms is updated to match the specifications of the ‘User traffic per wiki’ field
- The title of the ‘Sampling’ section is updated to 'Traffic and variations' in the experiment form, and to 'Traffic' in the instrument form
