https://manual.limesurvey.org/LimeSurvey-compatible_hosting_companies

@Jaredzimmerman (WMF)

In order to gather in context feedback on functionality in the past, we have had feedback links that take users directly to a survey tool (for example the feedback survey link in media viewer - with survey monkey). If we have to ask people to go to a wiki page and explain that we are taking them to another place for them to fill out the survey, the amount of fast, in context feedback we get will be a lot less than if we can just take people to the survey directly.

All that said, I am totally in favor of using open source tools when they are available and functional for the way we need to use them.
I have looked at Lime and as far as I can tell it is not nearly as advanced a survey tool as Qualtrics, and it is labor intensive to set up and use. I do not have engineering resources to build Lime to the place we need it to be to be an effective tool for the user research team. If that changes, and someone can provide those resources, I would be very happy to use Lime as it is open source. Until then, we need to use a survey tool that provides the functionality and usability we need right now. We want to understand the experience of, and gather feedback on functionality from all of the users. Many of our users (many readers and new / casual editors) are not familiar with the internal wiki feedback mechanisms, so we are attempting to reach out more broadly to gather a wider range of feedback. These types of tools help us do that.

We should figure out the "labor intensive to use" part, but the setup part can be fixed. :) We're trying to start working on it tomorrow, thanks to Ori's help.

Change 213579 had a related patch set uploaded (by Nemo bis):
[WIP] Stub LimeSurvey configuration

https://gerrit.wikimedia.org/r/213579

In T94807#1174681, @aripstra wrote:

FYI - Legal has vetted Qualtrics and it has gone through the Contracts team.

Since this task is about "Identify survey services compatible with our privacy policy", we can add Qualtrics to the list.

If I remember correctly, Community-Relations-Support has used SurveyMonkey, and #Engineering-Community uses Google Forms for i.e. event surveys. Are these services compatible with our privacy policy?

Principles that could guide next steps:

1. Assess the compatibility of the survey services used by the WMF with our policies.
2. Stop using survey services that are not compatible with our policies.
3. Consolidate the use of different services in one (exceptions should be possible, with a justification).
4. Investigate open source alternatives and propose and go for one if the feature set, sustainability of the project, and maintenance by SRE is clear.

Fabrice used SurveyMonkey for the Media Viewer survey, and I believe that it's been used by other projects in the past.

My team tested AllOurIdeas at the end of 2014. You can read a summary of its advantages and disadvantages here: https://meta.wikimedia.org/wiki/Community_Engagement_(Product)/Product_Surveys#Quarter_2_testing_phase

We are trying to help all open tasks listed under "Work continues after Lyon" at the Wikimedia Hackathon 2015 workboard finding their best way forward.

• If you are participating in Wikimania, consider adding the #Wikimania-Hackathon-2015 project to get this task in that loop, which is about to start.
• If you think this project could welcome help from a dedicated Google Summer of Code or Outreachy intern, or from an Individual Engagement Grant, add the Possible-Tech-Projects project.
• If you would like to receive some other type of support (organizing a Tech Talk, establishing contacts with existing developer teams in Wikimedia or elsewhere, travel sponsorship for a related activity... you name it), please create a subtask explaining your request and associate it with #Engineering-Community (or you can start by commenting here if you prefer).
• Keeping the description, priority and assigned fields up to date always helps. :)

For some context about this message, see T101151: Evaluate which projects showcased at the Wikimedia Hackathon should be supported further. It is the last communication related to Wikimedia-Hackathon-2015 that we will post here.

fwiw, survey by the Design team at https://wikimedia.qualtrics.com/jfe/form/SV_e9zX99TJlwGVYMt

It says at the top:

Your privacy is important to us. We will only share your responses with Wikimedia staff and contractors and in accordance with our privacy policy. We will only publicly share your responses or survey results in aggregate or anonymized form. By answering these questions, you permit us to record and indefinitely retain your responses for research purposes.

I see I never responded to this - pardon!

AllOurIdeas has been used by Liaisons, and may be used again. I had it cleared by Legal, but if there are concerns about it, do let me know.

So we have Qualtrics and AllOurIdeas that have been explicitly vetter by WMF-Legal. Is this enough to consider this task resolved, or what is missing?

For some (smaller) purposes, T89970: Enable microsurveys for long-term tracking of editing experience would solve the privacy problem: if we do it all in-house, then it's all subject to our own privacy policy and practices.

Also, informal on-wiki surveys are also possible. However, when my team offered the same survey both on-wiki and via Qualtrics last year, fewer than 1% of the respondents chose to reply on wiki.

The second point is very valuable @Whatamidoing-WMF.

@Qgil Manprit from Legal is looking at Google forms for surveys. Let's keep this task open at least until Legal is done with vetting (or not) Google forms. I'll update here what the result is.

Thank you! @Rfarrand, #engineering-community has been using Google Forms regularly. Let's try to find better alternatives like the ones described above the next time we need a registration/survey.

ECT will probably (mostly) move to Qualtrics soon as well, it is likely that we will stick with google forms for WMF internal event sign up and polling. @JAnstee_WMF from Learning and Evaluation recommends it highly and will be training me a bit on its use.
Any discussions about WMF surveys should probably include her as well! Adding her to the task. :)

Looking at the description of this task, I would say that we have almost completed it. We need to document its conclusions in a better place, though. Where?

https://meta.wikimedia.org/wiki/Survey_best_practices ?

I am interested in using an alternative service for more complex user surveys - Qualtrics is great but is expensive, so we are looking for other avenues.

Is Google Forms ok for user surveys? Is Lime Survey still being vetted?

Any updates on those points is appreciated!

@pizzzacat: if you need legal to review for a survey conducted by WMF, please email me directly and we can vet Google Forms or Live Survey. Thanks!

I'm working on gathering resources for surveys, including providing comparisons across different survey software available that we have experience with. Will post link here when its ready.

@Slaporte I'm terrible at Phabricator and just saw this. Thank you, I will need review on a survey in a week or two.

Given the end of Safe Harbor in few weeks from now (http://www.europarl.europa.eu/sides/getDoc.do?pubRef=-//EP//TEXT+TA+P8-TA-2015-0388+0+DOC+XML+V0//EN ), the legal considerations are soon outdated. For instance, the current harassment survey banner links a USA corporation without warning EU users that they will be transferring private data to the USA.

Upon clicking the survey, the participant should be greeted with a legal disclaimer that includes the language "By answering our questions, you consent to the transfer of your responses to the United States and other places as may be necessary to carry out the objectives of this project." and gives the option of exiting the survey if the participant is not ok with such a transfer. Hope this helps.

Just became aware of this task and thought I’d add a little information that I hope will be useful. The task is right - both Qualtrics and allourideas.org have been vetted by Legal. When we review a service in a circumstance like this, we go over their privacy policy, terms of service, and any individual contract we have with the third party in question, to make sure that the protections they offer user data are largely in line with our own. However, we also recognize that we have no control over how they may conduct themselves at all times and under all circumstances. Therefore, in the survey context, when we conduct a survey using a third-party service, we provide notice to survey-takers that the survey is on a third-party website, and we also provide links to the third party’s terms of service and privacy policy. When volunteers run a survey using a WMF-associated account, such as through Qualtrics, we ask them to provide the same notice and links. That way, survey takers can be informed of where/how their information is being handled.

Right now, Legal is working on updates to the survey toolbox on Meta, which will include some basic information about conducting surveys, including the use of third-party services. I will post to this ticket when the toolbox is complete, so all interested parties can take a look.

(I'm commenting since this page is linked from Meta-wiki FLOSS-Exchange and more historical context may be useful, since this Task mention Google Form as suggested platform.)

Since 2021 Wikimedia Foundation is using a LimeSurvey service. This was an indication from Legal, to mitigate risks related to Google Forms.

https://wikimediafoundation.limesurvey.net/

https://meta.wikimedia.org/wiki/LimeSurvey