Page MenuHomePhabricator

Welcome survey: turn off free text responses
Closed, ResolvedPublicJul 8 2019

Description

The objective of the free text options in the welcome survey was to learn what multiple-choice options we would need to include to cover as many user needs as possible. We have learned a bit from the free text options in the welcome survey after doing the analysis in T216081. Since doing that analysis, we have collected a lot more free text responses -- enough for us to complete the analysis and improve the multiple-choice options. Since we don't need more free text responses, we should turn off the ability to add them.

Specifically:

  • For the first question ("Why did you create your account today?") we should have the option "Other", but it should no longer read "Other (please describe)". It should just read "Other".
  • For the third question about topics of interest, the typeahead menu should not allow users to add their own responses -- they should only be able to choose from the list that pops up. Does this change have a particular effect on no-JS users?`

This change impacts Czech and Korean languages. The message is Welcomesurvey-question-reason-option-other-no-freetext-label. Translations completion:

  • cs
  • ko

Details

Due Date
Jul 8 2019, 7:00 AM
Related Gerrit Patches:
mediawiki/extensions/GrowthExperiments : masterAdd tests for freetext test feature flag
mediawiki/extensions/GrowthExperiments : masterWelcomeSurvey: Make freetext responses feature-flagged
mediawiki/extensions/GrowthExperiments : masterWelcomeSurvey: Add i18n messages for non-freetext "Other" option

Event Timeline

MMiller_WMF moved this task from Inbox to Upcoming Work on the Growth-Team board.

This is not urgent, but would like to do it by July 5.

MMiller_WMF set Due Date to Jul 5 2019, 7:00 AM.Jun 18 2019, 6:22 PM
Restricted Application changed the subtype of this task from "Task" to "Deadline". · View Herald TranscriptJun 18 2019, 6:22 PM
MMiller_WMF renamed this task from Welcome survey: temporarily turn off free text responses to Welcome survey: turn off free text responses.Jun 18 2019, 6:23 PM
MMiller_WMF changed Due Date from Jul 5 2019, 7:00 AM to Jul 8 2019, 7:00 AM.Jul 3 2019, 4:54 PM

It should just read "Other".

We'll need a new translation string, unfortunately. Our current policy is not to merge code that requires new translation strings on Friday/Monday/Tuesday. If this could wait to roll out with the train on July 18, there would be enough time for the translations to be merged using normal processes.

For the third question about topics of interest, the typeahead menu should not allow users to add their own responses -- they should only be able to choose from the list that pops up. Does this change have a particular effect on no-JS users?`

No JS users won't see a textfield at all, they will just be able to check the existing boxes.

@kostajh and I discussed, and we will try to get the new string translated in time tomorrow morning.

@kostajh -- is that the way the behavior has always been for no JS users? Just the checkboxes and no way to add anything else?

Change 521353 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] (wip) Make freetext responses in welcome survey feature flagged

https://gerrit.wikimedia.org/r/521353

Change 521385 had a related patch set uploaded (by Catrope; owner: Catrope):
[mediawiki/extensions/GrowthExperiments@master] WelcomeSurvey: Add i18n messages for non-freetext "Other" option

https://gerrit.wikimedia.org/r/521385

This patch does cause one display issue with variation C (the WelcomeSurvey popup), but since that's disabled in production right now I don't think that that's an urgent problem.


The issue happens because the popup code decides whether to show multiselect questions as checkboxes or as an input based on whether free text responses are allowed for that question (see the check for if ( config.allowArbitrary ) in WelcomeSurveyDialog.prototype.buildMultiSelectWidget). This means that turning off free text responses causes the "other topics" question to display as checkboxes instead of as an input, and the checkboxes take up so much more space that the question column starts scrolling. The Back and Next buttons are absolutely positioned, and misbehave this way in every situation where the question column scrolls.

Change 521385 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] WelcomeSurvey: Add i18n messages for non-freetext "Other" option

https://gerrit.wikimedia.org/r/521385

is that the way the behavior has always been for no JS users? Just the checkboxes and no way to add anything else?

No JS users saw checkboxes and a freetext field in which they could enter comma separated values:

Change 521353 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] WelcomeSurvey: Make freetext responses feature-flagged

https://gerrit.wikimedia.org/r/521353

Trizek-WMF updated the task description. (Show Details)Jul 9 2019, 10:19 AM

Change 521490 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Add tests for freetext test feature flag

https://gerrit.wikimedia.org/r/521490

Etonkovidova added a comment.EditedJul 9 2019, 4:53 PM

@kostajh (and @MMiller_WMF ) users are still able to enter text in "Add more topics..." which causes display issues:

  • if no topic gets selected and only freetext entered, then the first available topic ("Entertainment") would be entered, the user-entered text would be discarded.
  • if one (or more) topics were selected (none of them is "Entertainment") , then the result of adding freetext would as in the case above: user-entered text would be discarded.
  • if a selected topic is "Entertainment" and a user enters text, then the entered text would be marked as red.


The submission of the survey seems to be fine.

if a selected topic is "Entertainment" and a user enters text, then the entered text is marked as red.

Seems like we may need allowDisplayInvalidTags to make this more clear to the user

Etonkovidova added a comment.EditedJul 9 2019, 5:31 PM

For a quick comparioson between disabled free text: betalabs and production.

betalabscurrent production

I agree with @Etonkovidova that the automatic selection of "Entertainment" should be fixed. I think the best interaction would be if a user enters text that isn't allowed, it shows in red and disappears when they click away. Is that possible?

Change 521490 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Add tests for freetext test feature flag

https://gerrit.wikimedia.org/r/521490

I agree with @Etonkovidova that the automatic selection of "Entertainment" should be fixed. I think the best interaction would be if a user enters text that isn't allowed, it shows in red and disappears when they click away. Is that possible?

This is how MenuTagMultiSelectWidget works when "allowArbitrary" is set to false. Try going to the demo page and type in "blah" then press tab; the first item is entered. I'll file a task with OOUI to see if it gets fixed in the library.

@MMiller_WMF - can we just disable that field entirely or to change the label text to "Click and select additional topics of interest"?

I see that @matmarex has already merged a fix, so I'll wait for that to be in production with the next train before resolving this task.

Assuming that an OOUI release happens this week as usual, the fix will actually be in the second-next train (23-25 July rather than 16-18).

@matmarex - yes, the issue in T227666 seems to be fixed.

Moving to PM Review column to be checked after deployment.

MMiller_WMF closed this task as Resolved.Jul 22 2019, 11:42 PM

Thank you for completing this task!