@Catrope, @Krinkle: Do you know if there is a reliable way to detect that a user supports JS from the server side?

The use case is: we have 2 versions of the welcome survey after account creation. The no-js version is a special page we redirect to and the js version is a popup shown on top of the original context (based on returnTo and returnToQuery) the user is being redirected to after account creation.

It's hard to do the usual JS-based enhancement in this case because we want to go to different pages based on JS support.

Change 472721 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/GrowthExperiments@master] [wip] Welcome survey variation C

https://gerrit.wikimedia.org/r/472721

In T207717#4735347, @SBisson wrote:

@Catrope, @Krinkle: Do you know if there is a reliable way to detect that a user supports JS from the server side?

I don't believe so.

The use case is: we have 2 versions of the welcome survey after account creation. The no-js version is a special page we redirect to and the js version is a popup shown on top of the original context (based on returnTo and returnToQuery) the user is being redirected to after account creation.

It's hard to do the usual JS-based enhancement in this case because we want to go to different pages based on JS support.

The best idea that I can come up with (which isn't great) is to do a server-side redirect to Special:WelcomeSurvey as currently done in variation A, then have early-running JS on that page that does a redirect in JS to the returnto page with an extra param (e.g. /wiki/Page_I_came_from?welcomesurvey=1), then have JS on that page display an overlay.

Change 476505 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/GrowthExperiments@master] Welcome survey C: UI messages

https://gerrit.wikimedia.org/r/476505

Change 476505 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Welcome survey C: UI messages

https://gerrit.wikimedia.org/r/476505

In the current version of the patch, the checkbox sometimes doesn't immediately go away when an option is unselected.

In T207717#4788210, @Catrope wrote:

In the current version of the patch, the checkbox sometimes doesn't immediately go away when an option is unselected.

I think what's happening here is that the checkbox is focused and that's why I see an outline, because something similar happens when selecting:

I assumed this was caused by the accessibility feature that forces everything to have a border when selected so I didn't bother trying to change it.

Change 472721 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Welcome survey variation C

https://gerrit.wikimedia.org/r/472721

Change 477365 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[operations/mediawiki-config@master] Configure a sample welcome survey experiment for kowiki in labs

https://gerrit.wikimedia.org/r/477365

Change 477365 merged by jenkins-bot:
[operations/mediawiki-config@master] Configure a sample welcome survey experiment for kowiki in labs

https://gerrit.wikimedia.org/r/477365

@RHo -

(1) there are minor differences in text format (bold) comparing to the mockup (https://wikimedia.invisionapp.com/share/PBOLQ4KX479#/screens/327292403), but it's consistent with variation A, so I think that it's ok:

mockup	variation C

(2) The mockup does not include 'Skip this survey' option on the last page of the survey -is it intentional for variation C ?

(3) Also, once an option was checked on "Why did you create your account today?" or "Have you ever edited Wikipedia?", it's impossible to un-check all options . Users are forced to have one option checked - it's different from variation A.

In T207717#4798534, @Etonkovidova wrote:

(2) The mockup does not include 'Skip this survey' option on the last page of the survey -is it intentional for variation C ?

The survey does not include the skip button on the last page. Your screenshots are showing the first and 3rd pages.

In T207717#4798534, @Etonkovidova wrote:

@RHo -
(1) there are minor differences in text format (bold) comparing to the mockup (https://wikimedia.invisionapp.com/share/PBOLQ4KX479#/screens/327292403), but it's consistent with variation A, so I think that it's ok:

mockup	variation C

• Yes, the text formatting is fine on the options, sorry the mock should have been updated.

(2) The mockup does not include 'Skip this survey' option on the last page of the survey -is it intentional for variation C ?

• Yes, last page doesn't require skip

(3) Also, once an option was checked on "Why did you create your account today?" or "Have you ever edited Wikipedia?", it's impossible to un-check all options . Users are forced to have one option checked - it's different from variation A.

• Oh I didn't realise var A allows unchecking a radio button group once selected, that shouldn't be the case. So var C is right.

Thx, @RHo.

@SBisson and @MMiller_WMF - couple of observations on the form display on mobile (they are consistent between iphone/Android and the screen sizes) :
(1) No wrapping for some options

(3) (not mobile) MinervaNeue skin - this is the only survey page that displays overlapping:

@RHo -- one thing I noticed is that the last page of the survey has a grayed-out "Next" button. That made me feel like I had to check "Yes" on the mentorship question in order to proceed. Maybe there should be no "Next" button on that page?

Relatedly, I found it strange that the "Finish" button appeared in the upper right, even though the "Skip this survey" button was in the bottom right. Maybe the "Finish" button should be in the bottom right? And should it be labeled "Submit" like in Variation A?

Also, what do you think about a larger font size for the question text? Sometimes I felt like the font size was small enough that I even missed that the question was being asked -- especially because the responses are outlined in blue boxes, giving them greater affordance than the question. It seems like there is plenty of space in the dialog for larger question font.

To add to @MMiller_WMF's questions
The contrast of the checkmark against blue background is ok?

In T207717#4799304, @Etonkovidova wrote:

To add to @MMiller_WMF's questions
The contrast of the checkmark against blue background is ok?

The checkmark should have been white. I made a mistake when copying the svg. I'm working on a patch to fix all (or most) of the visual issues noted above.

In T207717#4798772, @Etonkovidova wrote:

Thx, @RHo.

@SBisson and @MMiller_WMF - couple of observations on the form display on mobile (they are consistent between iphone/Android and the screen sizes) :
(1) No wrapping for some options

Fixed in upcoming patch

(2) (unnecessary?) ellipses. In a horizontal view, the message is fully displayed, of course.

It looks like the confirmation message header cannot be fully displayed on mobile. I can make it wrap but it makes the header taller and I have to vertically align the button and it snowballs into more changes and breaks the harmony of the header/footer. I'd rather not go there if we can avoid it.

Maybe it could be shorten to just "Thanks!" ( cc @RHo )

(3) (not mobile) MinervaNeue skin - this is the only survey page that displays overlapping:

I'm tempted to let this one go. It's extremely unlikely that a new user would be using MinervaNeue immediately after sign up.

Change 477780 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/GrowthExperiments@master] Welcome survey C: visual improvements

https://gerrit.wikimedia.org/r/477780

Change 477780 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Welcome survey C: visual improvements

https://gerrit.wikimedia.org/r/477780

In T207717#4798582, @SBisson wrote:

In T207717#4798534, @Etonkovidova wrote:

(2) The mockup does not include 'Skip this survey' option on the last page of the survey -is it intentional for variation C ?

The survey does not include the skip button on the last page. Your screenshots are showing the first and 3rd pages.

My screenshots refer to (1). And @RHo answered that not having 'skip' on the last page is fine.

In T207717#4800166, @SBisson wrote:

In T207717#4798772, @Etonkovidova wrote:

(3) (not mobile) MinervaNeue skin - this is the only survey page that displays overlapping:

I'm tempted to let this one go. It's extremely unlikely that a new user would be using MinervaNeue immediately after sign up.

I agree.

Responses to a few comments above plus a couple of design tweaks:

A. Confirmation message header

In T207717#4800166, @SBisson wrote:

It looks like the confirmation message header cannot be fully displayed on mobile. I can make it wrap but it makes the header taller and I have to vertically align the button and it snowballs into more changes and breaks the harmony of the header/footer. I'd rather not go there if we can avoid it.
Maybe it could be shorten to just "Thanks!" ( cc @RHo )

Do you mean update to "Thanks" in the dialog header for desktop and mobile? I'd prefer if we can instead move the "thanks" message to be in the main dialog instead and keep the existing dialog header as "Welcome, $username"

B. Minerva weirdness

In T207717#4800166, @SBisson wrote:

In T207717#4798772, @Etonkovidova wrote:

(3) (not mobile) MinervaNeue skin - this is the only survey page that displays overlapping:

I'm tempted to let this one go. It's extremely unlikely that a new user would be using MinervaNeue immediately after sign up.

I agree.

+1 since this seems unlikely

C. Placement of Next and Finish CTAs

In T207717#4799053, @MMiller_WMF wrote:

@RHo -- one thing I noticed is that the last page of the survey has a grayed-out "Next" button. That made me feel like I had to check "Yes" on the mentorship question in order to proceed. Maybe there should be no "Next" button on that page?
Relatedly, I found it strange that the "Finish" button appeared in the upper right, even though the "Skip this survey" button was in the bottom right. Maybe the "Finish" button should be in the bottom right? And should it be labeled "Submit" like in Variation A?

– I agree it is weird. @SBisson can we remove the disabled Next and place the Finish button there instead?

• @MMiller_WMF - I thought we had made it "Finish" in var A as well in the final copy?

D. Minor form and text styling improvements

In T207717#4799053, @MMiller_WMF wrote:
Also, what do you think about a larger font size for the question text? Sometimes I felt like the font size was small enough that I even missed that the question was being asked -- especially because the responses are outlined in blue boxes, giving them greater affordance than the question. It seems like there is plenty of space in the dialog for larger question font.

I think the standard form font sizes are OK, but we can make improvements to the surrounding secondary elements.
Here are some proposed tweaks:
i. Welcome survey subtitle - reduce font-size to be font-size: 14px // 1em), and make line-height: 1.5 for easier readability.
ii. Add more spacing above and below the stack indicator dots .stack-position-indicator { padding: 0.5em 0 1em; }
iii. Sidebar headers - reduce font-size to be 'font-size: .875em //14px`
iv. Sidebar paragraph text - make sparser line-height: 21px // 1.5 for easier readability and also text color should be 'color: #222` per standard <p> style (it looks to be #000 in the proto).

@MMiller_WMF - I thought we had made it "Finish" in var A as well in the final copy?

Yes, you're right. It is that way in Variation A. Sorry. I don't know why I thought it was "Submit".

When @Urbanecm was testing out Variation C in beta this morning, he brought up two questions for @RHo:

• On the very first screen (the question about why they made their account), there is a grayed-out back button. Since there are no circumstances in which that back button would not be grayed out, should we just not have a back button there?
• If a respondent is looking at a question and pushes "Next" without answering it, that is effectively them saying "Skip". Should the button literally say "Skip" until they give a response, at which point it becomes "Next"?

In T207717#4804012, @MMiller_WMF wrote:

When @Urbanecm was testing out Variation C in beta this morning, he brought up two questions for @RHo:

• On the very first screen (the question about why they made their account), there is a grayed-out back button. Since there are no circumstances in which that back button would not be grayed out, should we just not have a back button th

• Yes agree, can we hide "Back" on the first step @SBisson ?

• If a respondent is looking at a question and pushes "Next" without answering it, that is effectively them saying "Skip". Should the button literally say "Skip" until they give a response, at which point it becomes "Next"?

• Let's keep it as "Next" since it is confusing to have two actions that say Skip ("Skip this survey" and "Skip" to the next question)

@RHo I've made all the visual changes suggested above (patch coming up) but I'm not sure about "C. Placement of Next and Finish CTAs".

Putting "Finish" where "Next" is means that clicking "next" repeatedly has a high chance of the user hitting "Finish" by accident.

In T207717#4804051, @RHo wrote:

In T207717#4804012, @MMiller_WMF wrote:

When @Urbanecm was testing out Variation C in beta this morning, he brought up two questions for @RHo:

• On the very first screen (the question about why they made their account), there is a grayed-out back button. Since there are no circumstances in which that back button would not be grayed out, should we just not have a back button th

• Yes agree, can we hide "Back" on the first step @SBisson ?

This would mean "back" and "next" shifting under the user's cursor. I'll do it if you really insist but I see absolutely nothing wrong with navigation buttons being stable and enable/disable as relevant.

Change 478044 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/GrowthExperiments@master] Welcome survey C: UI fixes

https://gerrit.wikimedia.org/r/478044

Change 478044 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Welcome survey C: UI fixes

https://gerrit.wikimedia.org/r/478044

In T207717#4804166, @SBisson wrote:

@RHo I've made all the visual changes suggested above (patch coming up) but I'm not sure about "C. Placement of Next and Finish CTAs".

Putting "Finish" where "Next" is means that clicking "next" repeatedly has a high chance of the user hitting "Finish" by accident.

Hi @SBisson - I've just gone over the test again on Desktop and Mobile, and think we should definitely move the "Finish". We will see if there is a higher submission rate of blank responses, which I think is worth the risk over people clicking "skip" instead because they can't see the "Finish" button.

In T207717#4804167, @SBisson wrote:

This would mean "back" and "next" shifting under the user's cursor. I'll do it if you really insist but I see absolutely nothing wrong with navigation buttons being stable and enable/disable as relevant.

• I take your point about shifting the button position being a negative evening out the weirdness of having a disabled "Back". Not as fussed about updating this so fine to leave as is.

hi @SBisson - one more thing I noticed in the mobile test is that when tapping "Next", it proceeds to the next question without returning the user to the top of the screen. For example if I've scrolled to enter my own interest topics on Q3 and tap "Next", I do not see the "Enter an email" field because the viewport is on the lower part of the dialog. See vid showing the behavior here: https://youtu.be/RF9meXc7K0A?t=21 (start ~21sec mark)

Can tapping the "Next" always return users to the start of the next screen?

In T207717#4804051, @RHo wrote:

In T207717#4804012, @MMiller_WMF wrote:

When @Urbanecm was testing out Variation C in beta this morning, he brought up two questions for @RHo:

• On the very first screen (the question about why they made their account), there is a grayed-out back button. Since there are no circumstances in which that back button would not be grayed out, should we just not have a back button th

• Yes agree, can we hide "Back" on the first step @SBisson ?

• If a respondent is looking at a question and pushes "Next" without answering it, that is effectively them saying "Skip". Should the button literally say "Skip" until they give a response, at which point it becomes "Next"?

• Let's keep it as "Next" since it is confusing to have two actions that say Skip ("Skip this survey" and "Skip" to the next question)

When I was testing it I was gonna ask Marshall "wait, the questions are mandatory?", and I realized he told me before that they're optional several seconds after, so I proposed the button can say Skip in this case instead.

If this confused me for a while (as I consider myself as someone who knows a little how this should work), it can confuse a newbie too. The comparation of risk of having a newbie answering all questions because they think they are mandatory and a risk of having a newbie confused about the difference of Skip question and Skip survey is up to you of course, I'm just sharing my thoughts.

We can minimalize the second kind of risk by having the button say "Skip this question" instead. Of course, that can make significant differences in length of that button. To solve this, we can have the button say only Skip this question and show next question when user pickes a response.

In T207717#4804167, @SBisson wrote:

In T207717#4804051, @RHo wrote:

In T207717#4804012, @MMiller_WMF wrote:

When @Urbanecm was testing out Variation C in beta this morning, he brought up two questions for @RHo:

• On the very first screen (the question about why they made their account), there is a grayed-out back button. Since there are no circumstances in which that back button would not be grayed out, should we just not have a back button th

• Yes agree, can we hide "Back" on the first step @SBisson ?

This would mean "back" and "next" shifting under the user's cursor. I'll do it if you really insist but I see absolutely nothing wrong with navigation buttons being stable and enable/disable as relevant.

Or that the next button stays at current place (so there would be blank space before it). Anyway, to clarify, I meant to raise only the second question Marshall wrote, even I recall talking about the Back button, I have no problems with having it disabled on the first screen. I sadly don't remember why I talked about it...

Change 478218 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/GrowthExperiments@master] Welcome survey C: scroll to the top when navigating between questions

https://gerrit.wikimedia.org/r/478218

Change 478230 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/GrowthExperiments@master] Welcome survey C: show/hide nav buttons, move publish btn into toolbar

https://gerrit.wikimedia.org/r/478230

Change 478218 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Welcome survey C: scroll to the top when navigating between questions

https://gerrit.wikimedia.org/r/478218

Change 478230 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Welcome survey C: show/hide nav buttons, move publish btn into toolbar

https://gerrit.wikimedia.org/r/478230

Checked https://gerrit.wikimedia.org/r/478230 in betalabs - all looks good.

In T207717#4805952, @Urbanecm wrote:

When I was testing it I was gonna ask Marshall "wait, the questions are mandatory?", and I realized he told me before that they're optional several seconds after, so I proposed the button can say Skip in this case instead.
If this confused me for a while (as I consider myself as someone who knows a little how this should work), it can confuse a newbie too. The comparation of risk of having a newbie answering all questions because they think they are mandatory and a risk of having a newbie confused about the difference of Skip question and Skip survey is up to you of course, I'm just sharing my thoughts.
We can minimalize the second kind of risk by having the button say "Skip this question" instead. Of course, that can make significant differences in length of that button. To solve this, we can have the button say only Skip this question and show next question when user pickes a response.

I think the risk of confusion by renaming the first button "Skip this question" along with “Skip this survey” outweighs the minor risk of people thinking that the survey questions are mandatory (since we do say they are optional in the intro text on that same first question). Secondly it will substantial effort for the change to auto-advance to the next question when the user picks an answer, not to mention the need for an additional “next” button if the user picks “Other” as the response.
My recommendation is we go with the survey as it is now and monitor response rate. If we see a great decline in completion by people exiting at this first step then we can re-evaluate.

LGTM on Desktop and Mobile

@SBisson @RHo @Etonkovidova -- thank you for making this turn out great!

I have a few questions and issues from trying this on mobile in Safari on my iPhone:

• I found it challenging to scroll up and down in the survey correctly. As I tried to scroll, I could see the scroll bar move up and down on the right, but the survey wasn't moving. I think what was happening was I was accidentally scrolling the page behind the survey, the one that is being overlaid on top of.
• I had similar challenges clicking "Next" and "Back". It's like the first time I touched those buttons, the focus wasn't on them or something, so I would have to click them a second time to actually get it to go through.
• I noticed that the email address question and the mentorship question are on the same page on both mobile and desktop. Is that deliberate? Do we prefer it that way instead of separated?

• @RHo -- why do we want it so that users can't unselect a radio button? To encourage them to submit their response?

In T207717#4815994, @MMiller_WMF wrote:

@SBisson @RHo @Etonkovidova -- thank you for making this turn out great!

I have a few questions and issues from trying this on mobile in Safari on my iPhone:

• I noticed that the email address question and the mentorship question are on the same page on both mobile and desktop. Is that deliberate? Do we prefer it that way instead of separated?

• hi Marshall, these have always been on the same step in the design since they are somewhat related. The other more pertinent reason is that for users who provided their email during account sign-up don't see the email field, only the mentor checkbox question. Having them on same step means not having to have a conditionally shown extra 5th step.

• @RHo -- why do we want it so that users can't unselect a radio button? To encourage them to submit their response?

Not so much that, but more so that is the way single-selection response type questions are by design. I.e., the group of radio buttons offers the gamut of all responses possible, for which users choose 1 response. So if someone has made a selection by accident on option A, they should still be able to select whatever the pertinent answer for them in that list.
FWIW, it is the same radio button control as var A, it has just been styled differently – so in var A as well if someone had selected an option by accident they would not have been able to deselect all either.

In T207717#4816636, @RHo wrote:

[...]
FWIW, it is the same radio button control as var A, it has just been styled differently – so in var A as well if someone had selected an option by accident they would not have been able to deselect all either.

Var A is using dropdowns instead of radio groups for the first 2 questions and they include a placeholder value ("Please select...") that the user can go back to.

In T207717#4816789, @SBisson wrote:

In T207717#4816636, @RHo wrote:

[...]
FWIW, it is the same radio button control as var A, it has just been styled differently – so in var A as well if someone had selected an option by accident they would not have been able to deselect all either.

Var A is using dropdowns instead of radio groups for the first 2 questions and they include a placeholder value ("Please select...") that the user can go back to.

Oh derp, sorry you're right I was looking at the earlier version.

In T207717#4815994, @MMiller_WMF wrote:

@SBisson @RHo @Etonkovidova -- thank you for making this turn out great!

I have a few questions and issues from trying this on mobile in Safari on my iPhone:

• I found it challenging to scroll up and down in the survey correctly. As I tried to scroll, I could see the scroll bar move up and down on the right, but the survey wasn't moving. I think what was happening was I was accidentally scrolling the page behind the survey, the one that is being overlaid on top of.

This is concerning. I did not experience that with Chrome's device simulator or Firefox on my Android phone.

If you go to this page https://doc.wikimedia.org/oojs-ui/master/demos/?page=dialogs&theme=wikimediaui&direction=ltr&platform=desktop
and click the link that says "Process dialog (medium, long)", are you seeing the same issue?

• I had similar challenges clicking "Next" and "Back". It's like the first time I touched those buttons, the focus wasn't on them or something, so I would have to click them a second time to actually get it to go through.

This is annoying, and I also can't reproduce it with what I have. I'd be curious to know if you experience it with the dialogs on the demo page (link above).

@MMiller_WMF - I re-tested the following on iPhone 6S iOS version 12.1.1

In T207717#4815994, @MMiller_WMF wrote:

I have a few questions and issues from trying this on mobile in Safari on my iPhone:

• I found it challenging to scroll up and down in the survey correctly. As I tried to scroll, I could see the scroll bar move up and down on the right, but the survey wasn't moving. I think what was happening was I was accidentally scrolling the page behind the survey, the one that is being overlaid on top of.
• I had similar challenges clicking "Next" and "Back". It's like the first time I touched those buttons, the focus wasn't on them or something, so I would have to click them a second time to actually get it to go through.

The issues were not present after I've updated iOS. I saw the issues before on my device and they were not reproducible in any of browser simulators. I'll do more testing on a real Android phone shortly.

@SBisson @Etonkovidova @RHo -- I was able to replicate the issues in both Variation C and in the standard component library that @SBisson linked me to. I created videos of it, and they're in our team drive as Variation_C_iPhone_Safari_2018-12-14.mov and Standard_components_iPhone_Safari_2018-12-14.mov. I think you should view them in that order.

What do you think?

I'm on an iPhone 6S, running the latest version of iOS (12.1.1) and using Safari in private browsing mode.

(1) The content in the background will flash momentarily when scrolling down (not too much interfering)

(2) Pressing 'Next' is not working only the first time - all subsequent pressings are fine.

Overlap also happens on Vector when resizing the window: