Page MenuHomePhabricator

Organize a (annual?) toollabs survey
Closed, ResolvedPublic

Description

Currently the feedback is mostly negative because it's bugs or people complaining on the mailing list. We should do a survey in regular intervals to get user feedback on toollabs. It should focus on the people who have an account on toollabs, but the scope could of course be broader. The questions should be made in a way that the survey helps to figure out if toollabs is on the right path. Are the volunteer developers happy with toollabs? Probably good to gather some input on what questions to ask.

More information about the survey, when available, will be gathered in https://meta.wikimedia.org/wiki/Research:Annual_Tool_Labs_Survey .

The evaluation team (https://meta.wikimedia.org/wiki/Grants:Evaluation) can probably help out here or even take the lead.

todos:

itemstatuswho
collect first set of questions from YuvidoneLeila
prepare a first draft of the surveydoneLeila
ask for Yuvi's feedback on the draft and improve the surveydoneYuvi, Leila
ask for few employee feedback and improvedoneYuvi, Leila
make a page on research meta to capture the survey and resultsdoneLeila
ask for 2-3 volunteer feedback and improvedoneYuvi,Leila
run the survey by LegaldoneLeila
prepare a list of potential participantsdoneYuvi
send the survey out outdonescheduled for 2015-09-25
analysisscheduledafter survey end date 2015-10-08

See also

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Multichill raised the priority of this task from to Medium.Apr 6 2015, 8:01 AM
Multichill updated the task description. (Show Details)
Multichill added subscribers: Multichill, Rfarrand, QuimGil and 4 others.

@Multichill by "brother" you meant "broader", didn't you?

JAnstee_WMF added a subscriber: JAnstee_WMF.

Thanks for the tag, changed it to our team rather than specific reporting project task group.

@leila has graciously accepted to help run this one :)

Adding this to the Design research work board for awareness. Please reach out if you need feedback on your survey or anything.

A first set of questions collected from Yuvi last week. A first draft is ready for Yuvi's review now (Yuvi, check your email).

We have a few surveys going out this month, one for Grants consultation and another for Community Tech, which together are targeting many registered and very active users. It would be super helpful if someone could fill out this form to indicate scope and dates. We are trying to avoid burning out users with surveys.

https://docs.google.com/forms/d/1ughvbZgRzVvlTpA2pP5-LkdjzXuWQnRL0aSnLa51OpQ/viewform

Thanks!
Edward

@egalvezwmf Thanks for sharing the form. I did not know about it. It seems I should have asked you guys for helping us for the Tool Labs survey. ;-)
The form you shared is for when L&E designs surveys. I agree with you that it's important for teams to coordinate but filling out that long form when most of it doesn't apply to this case won't work. Are the dates and populations affected already shared in a wiki page? I'm happy to add the same information somewhere public everyone can see, and I support doing it.

@Capt_Swing based on our IRC conversation just now, you did not know about the form egalvezwmf shared above. Since your team runs surveys frequently, you may want to be aware of it.

@leila, @Capt_Swing, I just created the form yesterday, and will be a way for me to support WMF teams in designing their surveys and to help me figure out how much support we need to provide. Also part of my role this year will be to help coordinate surveys at the WMF and to prevent survey fatigue (likely a wiki page) but still planning this out. I will be talking about this more during the research and data meeting on thursday.

@leila, totally get it - what about just adding dates and/or sampling strategy to the info above?

Thanks!

I have cleared with Legal that we can use Google forms to run this survey. Two things we should do:

  1. Before we share the link, we should clearly indicate that the survey they are taking will be on Google and thus subject to the Google privacy policy.
  1. We should make it clear the survey is optional. We cannot use Google forms and leave the user with the impression that they have to participate.

Once we finalize the survey, we should still run it by Legal for their final word.

@yuvipanda, if you send me the number of users we will contact, I'll make the meta page today. If there is a special place you want the survey results and description sit, let me know. Otherwise, I'll make the page in https://meta.wikimedia.org/wiki/Research. Per earlier discussion, Monday late afternoon PST should be a good time for us to send out the survey. :-)

Oops, I've been flaky. I'll send this out on Wednesday... Apologies for the delay

leila updated the task description. (Show Details)

Looking forward to the analysis! Where is the analysis going to be posted? has this been documented in research:Meta?

@leila, @yuvipanda, wondering how this is going? Wondering if you need support or if its possible to post a brief summary of the results so far? I'd be happy to help with basic reporting of results if needed. Note that I will be on vacation starting end of next week. Thanks!

update: @bd808 has done a full analysis of the survey data. We are now finalizing the analysis and will share that, hopefully soon.

@Multichill bear with us. We're trying to get this to you soon, for sure before Wikimania this year. :">

update: @bd808 has done a full analysis of the survey data. We are now finalizing the analysis and will share that, hopefully soon.

@Multichill bear with us. We're trying to get this to you soon, for sure before Wikimania this year. :">

Before the Jerusalem hackathon would be nice if possible.

I just learned that's the plan. I do what I can to help make it happen @Multichill.

At long last the detailed results have been published on meta and an announcement sent to the labs-l and wikitech-l mailing lists.

Hi all - the tool labs survey is a great candidate for the new community engagement insights process, which is now posted on meta.

I am happy to help teams through this process. The Intent form is due on August 5th if your team will be participating. Make sure to spread the word as well. Note that questions are not due - just your team's intent to participate! A first draft of survey questions would be due the first week of September.