Page MenuHomePhabricator

Organize a (annual?) toollabs survey
Closed, ResolvedPublic

Description

Currently the feedback is mostly negative because it's bugs or people complaining on the mailing list. We should do a survey in regular intervals to get user feedback on toollabs. It should focus on the people who have an account on toollabs, but the scope could of course be broader. The questions should be made in a way that the survey helps to figure out if toollabs is on the right path. Are the volunteer developers happy with toollabs? Probably good to gather some input on what questions to ask.

More information about the survey, when available, will be gathered in https://meta.wikimedia.org/wiki/Research:Annual_Tool_Labs_Survey .

The evaluation team (https://meta.wikimedia.org/wiki/Grants:Evaluation) can probably help out here or even take the lead.

todos:

itemstatuswho
collect first set of questions from YuvidoneLeila
prepare a first draft of the surveydoneLeila
ask for Yuvi's feedback on the draft and improve the surveydoneYuvi, Leila
ask for few employee feedback and improvedoneYuvi, Leila
make a page on research meta to capture the survey and resultsdoneLeila
ask for 2-3 volunteer feedback and improvedoneYuvi,Leila
run the survey by LegaldoneLeila
prepare a list of potential participantsdoneYuvi
send the survey out outdonescheduled for 2015-09-25
analysisscheduledafter survey end date 2015-10-08

See also

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Multichill raised the priority of this task from to Normal.Apr 6 2015, 8:01 AM
Multichill updated the task description. (Show Details)
Multichill added subscribers: Multichill, Rfarrand, QuimGil and 4 others.
scfc moved this task from Triage to Backlog on the Toolforge board.Apr 6 2015, 8:12 AM

@Multichill by "brother" you meant "broader", didn't you?

Multichill updated the task description. (Show Details)Apr 6 2015, 10:11 AM
Multichill set Security to None.
JAnstee_WMF added a subscriber: JAnstee_WMF.

Thanks for the tag, changed it to our team rather than specific reporting project task group.

Restricted Application added a project: Cloud-Services. · View Herald TranscriptJul 15 2015, 8:44 PM
leila claimed this task.Jul 15 2015, 9:07 PM

@leila has graciously accepted to help run this one :)

mcruzWMF added a subscriber: egalvezwmf.

Adding this to the Design research work board for awareness. Please reach out if you need feedback on your survey or anything.

QuimGil edited subscribers, added: Qgil; removed: QuimGil.Jul 24 2015, 8:19 PM
leila added a comment.Aug 4 2015, 7:11 PM

A first set of questions collected from Yuvi last week. A first draft is ready for Yuvi's review now (Yuvi, check your email).

leila moved this task from Staged to In Progress on the Research board.Aug 4 2015, 7:12 PM
egalvezwmf added a comment.EditedAug 4 2015, 7:16 PM

We have a few surveys going out this month, one for Grants consultation and another for Community Tech, which together are targeting many registered and very active users. It would be super helpful if someone could fill out this form to indicate scope and dates. We are trying to avoid burning out users with surveys.

https://docs.google.com/forms/d/1ughvbZgRzVvlTpA2pP5-LkdjzXuWQnRL0aSnLa51OpQ/viewform

Thanks!
Edward

leila updated the task description. (Show Details)Aug 4 2015, 7:17 PM
leila added a comment.Aug 4 2015, 7:32 PM

@egalvezwmf Thanks for sharing the form. I did not know about it. It seems I should have asked you guys for helping us for the Tool Labs survey. ;-)
The form you shared is for when L&E designs surveys. I agree with you that it's important for teams to coordinate but filling out that long form when most of it doesn't apply to this case won't work. Are the dates and populations affected already shared in a wiki page? I'm happy to add the same information somewhere public everyone can see, and I support doing it.

@Capt_Swing based on our IRC conversation just now, you did not know about the form egalvezwmf shared above. Since your team runs surveys frequently, you may want to be aware of it.

@leila, @Capt_Swing, I just created the form yesterday, and will be a way for me to support WMF teams in designing their surveys and to help me figure out how much support we need to provide. Also part of my role this year will be to help coordinate surveys at the WMF and to prevent survey fatigue (likely a wiki page) but still planning this out. I will be talking about this more during the research and data meeting on thursday.

@leila, totally get it - what about just adding dates and/or sampling strategy to the info above?

Thanks!

leila updated the task description. (Show Details)Aug 4 2015, 8:19 PM
leila updated the task description. (Show Details)Aug 7 2015, 12:05 AM
leila added a comment.Aug 7 2015, 2:47 AM

I have cleared with Legal that we can use Google forms to run this survey. Two things we should do:

  1. Before we share the link, we should clearly indicate that the survey they are taking will be on Google and thus subject to the Google privacy policy.
  1. We should make it clear the survey is optional. We cannot use Google forms and leave the user with the impression that they have to participate.

Once we finalize the survey, we should still run it by Legal for their final word.

leila updated the task description. (Show Details)Aug 10 2015, 6:11 PM
leila moved this task from In Progress to Paused on the Research board.Aug 17 2015, 8:30 PM
leila moved this task from Paused to In Progress on the Research board.Aug 20 2015, 10:30 PM

CommTech is also running a survey. Sharing this in case the questions are helpful: https://meta.wikimedia.org/wiki/Research:Tech_support_satisfaction_poll/Questions

egalvezwmf moved this task from Backlog to Active on the Surveys board.Aug 27 2015, 3:06 PM
leila moved this task from In Progress to Paused on the Research board.Sep 10 2015, 9:19 PM
leila updated the task description. (Show Details)Sep 12 2015, 2:41 AM
leila moved this task from Paused to In Progress on the Research board.Sep 15 2015, 11:16 PM
leila updated the task description. (Show Details)Sep 18 2015, 5:29 PM
leila added a comment.Sep 18 2015, 5:31 PM

@yuvipanda, if you send me the number of users we will contact, I'll make the meta page today. If there is a special place you want the survey results and description sit, let me know. Otherwise, I'll make the page in https://meta.wikimedia.org/wiki/Research. Per earlier discussion, Monday late afternoon PST should be a good time for us to send out the survey. :-)

Oops, I've been flaky. I'll send this out on Wednesday... Apologies for the delay

leila updated the task description. (Show Details)Sep 22 2015, 10:42 PM
leila updated the task description. (Show Details)Sep 24 2015, 9:47 PM
leila updated the task description. (Show Details)
leila updated the task description. (Show Details)Sep 26 2015, 5:54 AM
leila updated the task description. (Show Details)Sep 26 2015, 5:57 AM
leila moved this task from In Progress to Paused on the Research board.Oct 1 2015, 10:03 PM
yuvipanda moved this task from To do to Done on the labs-sprint-117 board.Oct 9 2015, 10:44 AM

This closes tomorrow!

Looking forward to the analysis! Where is the analysis going to be posted? has this been documented in research:Meta?

leila moved this task from Paused to In Progress on the Research board.Oct 22 2015, 5:09 AM

@leila, @yuvipanda, wondering how this is going? Wondering if you need support or if its possible to post a brief summary of the results so far? I'd be happy to help with basic reporting of results if needed. Note that I will be on vacation starting end of next week. Thanks!

leila added a subscriber: bd808.Mar 3 2016, 7:41 PM

update: @bd808 has done a full analysis of the survey data. We are now finalizing the analysis and will share that, hopefully soon.

@Multichill bear with us. We're trying to get this to you soon, for sure before Wikimania this year. :">

Qgil moved this task from To triage to Team radar on the Developer-Advocacy board.
leila reassigned this task from leila to bd808.Mar 3 2016, 9:51 PM
Restricted Application added a project: User-bd808. · View Herald TranscriptMar 3 2016, 9:51 PM

update: @bd808 has done a full analysis of the survey data. We are now finalizing the analysis and will share that, hopefully soon.
@Multichill bear with us. We're trying to get this to you soon, for sure before Wikimania this year. :">

Before the Jerusalem hackathon would be nice if possible.

leila added a comment.Mar 3 2016, 10:12 PM

I just learned that's the plan. I do what I can to help make it happen @Multichill.

bd808 moved this task from To Do to In Dev/Progress on the User-bd808 board.Mar 3 2016, 10:47 PM
ggellerman moved this task from Backlog to Radar on the Research-Backlog board.Mar 17 2016, 10:28 PM
bd808 moved this task from Backlog to Ready on the Community-Tech-Tool-Labs board.Mar 25 2016, 2:15 AM
bd808 moved this task from Ready to Doing on the Community-Tech-Tool-Labs board.Mar 25 2016, 2:47 PM
bd808 closed this task as Resolved.May 5 2016, 2:03 AM

At long last the detailed results have been published on meta and an announcement sent to the labs-l and wikitech-l mailing lists.

bd808 moved this task from In Dev/Progress to Done on the User-bd808 board.May 6 2016, 12:07 AM
bd808 moved this task from Done to Archive on the User-bd808 board.May 12 2016, 3:58 PM
bd808 moved this task from Doing to Done on the Community-Tech-Tool-Labs board.May 16 2016, 3:29 PM
egalvezwmf added a comment.EditedJul 19 2016, 11:52 PM

Hi all - the tool labs survey is a great candidate for the new community engagement insights process, which is now posted on meta.

I am happy to help teams through this process. The Intent form is due on August 5th if your team will be participating. Make sure to spread the word as well. Note that questions are not due - just your team's intent to participate! A first draft of survey questions would be due the first week of September.