I emailed Edward asking about survey infra options.

Response: Qualtrics is the only thing that we have an account with (at least that Edward's team has an account with). He suggested google forms can be good if done well.

I have asked QA mailing list for comments.

Did another pass, left a XXX regarding the "getting started" documentation question (which one? link to it).

• Free text

I would be great to include some free text box fields to provide context about some
answers. For instance, personally "I would like to help with fixing failed tests" AND
"I would like to pair with somebody" BUT.. there's a but (time zones and scheduling are proving troublesome for me) :-)

• Consider using the word "would" in your questions.

"I prefer writing end-to-end ('browser') tests using this language"
This is a strange question for me. I would prefer writing browser tests in JS but I have never written browser tests in JS.

• Also consider asking questions around platform stability / infrastructure.

In T131123#2378111, @Jdlrobson wrote:

• Free text

I would be great to include some free text box fields to provide context about some
answers. For instance, personally "I would like to help with fixing failed tests" AND
"I would like to pair with somebody" BUT.. there's a but (time zones and scheduling are proving troublesome for me) :-)

Are you saying every question should have a free form answer? Or just some of the questions? Which ones?

• Consider using the word "would" in your questions.

I am not sure what you mean.

"I prefer writing end-to-end ('browser') tests using this language"
This is a strange question for me. I would prefer writing browser tests in JS but I have never written browser tests in JS.

We (Release-Engineering-Team) would provide tools and training, like we did for Ruby.

• Also consider asking questions around platform stability / infrastructure.

There are a few questions like that:

• Gerrit triggered Selenium+Jenkins jobs (after patch set submission)
  • are stable enough
  • are fast enough
• Time triggered Selenium+Jenkins jobs (daily)
  • are stable enough
  • are fast enough

Can you provide example(s)?

The form is created. Please take a look and let me know if anything needs to be fixed. If you want to edit the form yourself, let me know and I will give you access.

nitpick/bikeshed: I'm actually thinking that a 5-point scale (vs 3) would be better. 3 is nice and quick for people but 5 gives them a little more wiggle room and, I believe, will increase the chances that we see change (good or bad) in a follow up survey. We used a 7-point scale for the MW-Vagrant survey before: https://www.mediawiki.org/wiki/MediaWiki-Vagrant/Surveys/2014Q4

@greg agreed, changed to 5-point scale.

I was browsing the surveys tag and saw this one. Some suggestions/thoughts:

• Use the grid option in google forms; you can combine more than one question this way
• Check out these response scales. The smileys could be misleading/confusing. especially for the "I like.." questions; this could create bias in your responses. You could use the google form number scales, but its better to anchor each number with a word. If you want to still use smileys, you should change the questions to: "How do you feel about...." instead of "I like". You should probably also use upright smileys rather than sideways ones...

Thanks @egalvezwmf !

In T131123#2384340, @egalvezwmf wrote:

I was browsing the surveys tag and saw this one. Some suggestions/thoughts:

Thanks @egalvezwmf!

• Use the grid option in google forms; you can combine more than one question this way

Thanks, I was not aware of this feature.

• Check out these response scales. The smileys could be misleading/confusing. especially for the "I like.." questions; this could create bias in your responses. You could use the google form number scales, but its better to anchor each number with a word. If you want to still use smileys, you should change the questions to: "How do you feel about...." instead of "I like".

Thanks, I will think about it.

You should probably also use upright smileys rather than sideways ones...

Should I copy/paste unicode characters from here? http://unicode.org/emoji/charts/full-emoji-list.html

Is there a way to test support for characters? Greg's machine did not display a couple correctly.

Ok, I think I have done everything suggested. I have used sideways emoji (:)) instead of upright () because I did not have the time to test how will unicode emoji display on various platforms. At least @greg had trouble with displaying some unicode emoji.

As far as I am concerned, we can launch the survey. We might wait a week since next week is wikimania.

In T131123#2388757, @zeljkofilipin wrote:

As far as I am concerned, we can launch the survey. We might wait a week since next week is wikimania.

At this point, yeah, that makes sense.

In T131123#2379058, @zeljkofilipin wrote:

In T131123#2378111, @Jdlrobson wrote:

• Free text

I would be great to include some free text box fields to provide context about some
answers. For instance, personally "I would like to help with fixing failed tests" AND
"I would like to pair with somebody" BUT.. there's a but (time zones and scheduling are proving troublesome for me) :-)

Are you saying every question should have a free form answer? Or just some of the questions? Which ones?

The more the better (all optional). I would argue the more open ended your questions are the better the feedback you'd get.
But yes definitely for this question (any further comments/explain your answer).

• Consider using the word "would" in your questions.

I am not sure what you mean.

Compare:

1. "I prefer writing end-to-end ('browser') tests using this language"
2. "I would prefer writing end-to-end ('browser') tests using this language"

My answers for these would be different. I have never written browser tests in JS. I would probably prefer it but the first point suggests that I have experience. My suggestion is to be more careful with the wording here.

In T131123#2402902, @Jdlrobson wrote:

The more the better (all optional). I would argue the more open ended your questions are the better the feedback you'd get.
But yes definitely for this question (any further comments/explain your answer).

Every section now has optional comments.

Compare:

1. "I prefer writing end-to-end ('browser') tests using this language"
2. "I would prefer writing end-to-end ('browser') tests using this language"

My answers for these would be different. I have never written browser tests in JS. I would probably prefer it but the first point suggests that I have experience. My suggestion is to be more careful with the wording here.

Good point. I will change the questions.

The survey is sent.

Hi @zeljkofilipin. Could you add this survey to the community engagement calendar? https://meta.wikimedia.org/wiki/Community_Engagement/Calendar

Thanks, J

What's the privacy policy for this survey?

Good question, Risker.

All submissions will be seen and collated by members of the Release Engineering team (namely Zeljko and I).

Submissions do not require logging into Google nor providing a name or email.

A summarized version will be made public (on mw.org). The 1-5 ratings will be averaged and shared. The free form comments will not be shared directly (iow: no direct quotes but instead paraphrasing one or more at a time).

Does that help?

I or Zeljko will send out a reply with this above clarification on Monday (a US holiday, so unlikely) or Tuesday (more likely). I'm going to go back to my birthday today :)

In T131123#2422322, @Capt_Swing wrote:

Hi @zeljkofilipin. Could you add this survey to the community engagement calendar? https://meta.wikimedia.org/wiki/Community_Engagement/Calendar

Thanks, J

Thanks for the reminder, done!

https://meta.wikimedia.org/w/index.php?title=Community_Engagement%2FCalendar&type=revision&diff=15745621&oldid=15744765

In T131123#2423267, @Risker wrote:

What's the privacy policy for this survey?

Apologies for forgetting about that, this is my first survey. The privacy policy is here: https://wikimediafoundation.org/wiki/Survey_Privacy_Statement_for_July_2016_Browser_Testing_Satisfaction_Survey