Page MenuHomePhabricator

Create a survey that MediaWiki users can use to provide us with information
Closed, ResolvedPublic

Event Timeline

MarkAHershberger raised the priority of this task from to Needs Triage.
MarkAHershberger updated the task description. (Show Details)
MarkAHershberger changed Security from none to None.

Phabricator does have a polling and questionnaire application..

I don't know if there is a standard place to do this otherwise, but it's a pretty straight forward app that is in phab-01 to look at https://phab-01.wmflabs.org/vote/

Thanks for the pointer! That is helpful.

Need to develop the questions and find a place to host this survey.

@MarkAHershberger: Where was the scope of the survey discussed and where can that discussion be found (link)?
Currently it feels like no-one finding this task here could help because the scope feels a bit vague. Please clarify...

Broad outline of the survey:

  • How long have you been using MW?
  • What do you use it for?
    • Documentation?
    • Coordination?
  • How could MediaWiki distribution be improved?

@Qgil, how is this part of the Documentation project? The survey is of users of the MediaWiki software. Are you thinking the survey could pose questions like "Are you an [] admin [] MediaWiki hacker?" "How could MediaWiki documentation for admins / users / developers be improved?"

@cicalese , I put a page up on mwstake to start gathering these questions. I wanted to dump what was in my brain somewhere. Since this is your thing, feel free to delete it all. :)

http://mwstake.org/mwstake/wiki/MediaWiki_Usage_Questions

Once we have a survey put together how do we get it in front of MediaWiki using people? A few ideas, but I'd love some more.

  • Email to mediawiki, mediawiki-enterprise, semantic-mediawiki (dev and user), mailing lists
  • Sharing on social media
  • See if the folks at WikiApiary will help. Maybe a site banner or something for a week or two
  • See if the folks behind MediaWiki.org can help. Maybe a banner as well

T951 is related as the survey will further our understanding of who (and how) is using MediaWiki.

Ckoerner updated the task description. (Show Details)

Given that our budget is 0, I propose we use a Google Doc to capture the information. Any alternative suggestions?

We are using Google Forms for our hackathon surveys, so nothing against this option. Still, you might want to check alternatives at T94807: Identify survey services compatible with our privacy policy.

Given that Wikimania is a scant two weeks away I spoke to @cicalese and offed to help finish the survey work. I have a draft of the questions here and need your feedback!

http://mwstake.org/mwstake/wiki/MediaWiki_Usage_Questions

Once I have a few people (I'm looking at you!) review these questions and the language around them I will create a working survey.

As I mentioned earlier (T1114#1341595) I have some suggestions on the distribution of the survey.

  • What other opportunities exist that we can leverage to distribute?
  • What do we need to do to get approval/permission to use the channels mentioned?
  • Does something like this need to be reviewed by the WMF in some capacity?

If we want to have a window for folks to respond we must act quickly. As of today we have two weeks until Wikimania begins. I would think getting this in front of Mediawiki users ASAP would provide us with more insightful and complete information than if we wait any longer.

Please review (and share) this draft. I propose a goal of sending this out by the end of this week - July 4th.

Left my comments on the talk page. One more, though: I feel like we don't want to bog dow the respondents just yet with too many questions. I think @RHeigl has worked on these beofe and it''d be good to know how we can make this as effective as possible.

Dang. You are absolutely correct. We're at 20 + questions. What if we paired that down to say 10? I'll ask you and anyone else reading - what are the big 10 questions we want to ask? Think more about the answers. What top 10 things about MediaWiki users do we want to know? How do we ask questions that provide those answers?

Regarding community surveys, this one from the Drupal community was insightful.

https://www.drupal.org/node/1399056

Of note, their survey was at around 13 questions, in addition to 4 'demographic' questions.

Ok, how about this as a proposal. I've removed a few questions to shorten things down to 12 questions. These are what I feel (which is neither right or wrong) on what are the important questions to ask this go round. This would assume that we keep the 4 demographic questions.

  1. How long have you been using MediaWiki?
  2. How many MediaWiki sites do you manage?
  3. Are your sites public or private?
  4. How many end users are there of your MediaWiki sites?
  5. What MediaWiki version are you currently using?
  6. How many extensions do you use?
  7. Of those extensions which are your Top Five?
  1. Why do you use MediaWiki?
  2. Do you make contributions to the MediaWiki community?
  3. What would you like to see improved in MediaWiki?
  4. What is the most important thing the Mediawiki community should be focused on?
  5. We're striving to increase the participation of MediaWiki users outside of Wikimedia Foundation-led initiatives. What do you think is the biggest barrier to participation?

This is a good short list. Perhaps the following two questions are similar enough to be combined into one:

3.What would you like to see improved in MediaWiki?
4.What is the most important thing the Mediawiki community should be focused on?

And, the following two questions are thematically related and could benefit from being beside each other:

2.Do you make contributions to the MediaWiki community?
5.We're striving to increase the participation of MediaWiki users outside of Wikimedia Foundation-led initiatives. What do you think is the biggest barrier to participation?

Perhaps 5 could be simplified to:

  1. What barriers, if any, do you think exist to participation in the MediaWiki community?

Thanks for pulling this together!

Thanks, Chris, for this list, this is good stuff.

A few comments that come to my mind:

  1. What MediaWiki version are you currently using?

I think it is more interesting how frequently they update. So the question could be rephrased to

  1. When did you last update your MediaWiki? To which version (LTS version, newest release, newest patch version, or other?)

And the questions about extensions:

  1. How many extensions do you use?

I am not so sure why we would be interested in that. Maybe rephrase it:

  1. Did you modify your MediaWiki? How (add existing extensions, change skin, write new extensions)?

In addition to @cicalese's comment,

  1. What barriers, if any, do you think exist to participation in the MediaWiki community?

maybe we could ask how to help:

  1. What barriers, if any, do you think exist to participation in the MediaWiki community? What can we do to help?

@cicalese

3.What would you like to see improved in MediaWiki?
4.What is the most important thing the Mediawiki community should be focused on?

My thought was to have one question about the software itself and the other be about the community. Fishing for answers like, "I think VisualEditor should work on the Apple Newton." as opposed to, "I think there should be more inclusion of people from the southern hemisphere at events."

Does that make sense? Do you think I should reword those? How does this sound?

  • What would you like to see most improved in MediaWiki the software?
  • What would yo like to see most improved in the community around MediaWiki?

The other suggestions are spot on, thank you.

@Mglaser

When did you last update your MediaWiki? To which version (LTS version, newest release, newest patch version, or other?)

In the longer survey I had a separate question regarding frequency. How about we keep both questions to keep it quantitative.

How many extensions do you use?

I'm curious about extension usage. Is everyone using just WMF-led extensions? What's the usage of third-party extensions like?

How about, "Of the extensions you use, which are your Top Five?" And then have 5 blanks where they can enter the name of the extension?

Here's an updated draft based upon the feedback.

  1. How long have you been using MediaWiki?
  2. How many MediaWiki sites do you manage?
  3. Are your sites public or private?
  4. How many end users are there of your MediaWiki sites?
  5. What MediaWiki version are you currently using?
  6. How frequently do you update your MediaWiki installations and extensions?
  7. Of the extensions you use, which are your Top Five?
  1. Why do you use MediaWiki?
  2. What would you like to see most improved in MediaWiki the software?
  3. What would yo like to see most improved in the community around MediaWiki?
  4. Do you make contributions to the MediaWiki community?
  5. What barriers, if any, do you think exist to participation in the MediaWiki community? What can we do to help?

The survey has been created!

Please review. If you are OK with sending, please let me know. If you are not OK with sending, please let me know.

I will not publish or send this until I have at least 4 responses in the affirmative. I don't want to send it out with the assumption everyone is OK with it! :)

http://goo.gl/forms/XCYIdNT2gI

Might be worth separating people who are actually running this week's WMF branches from those who are running really old WMF branches.

Typo on "What would yo like to see most improved in the community around MediaWiki?"

And " I contribute patches and pug fixes"

On the third page, I am asked how many MediaWiki sites I run. I selected more than one. In the follwing question about my MW version, I have to pick one. However, I run a range of versions. This might probably be better if I could select multiple options here.

Otherwise it's ok for me

@Krenair Fixed! Thank you.

@Mglaser Hmm. Good point. I've changed it to a multi-select.

Checked it. Great!
You have my OK :-))

Thank you very much!

Just found a typo: Are you(r) sites public or private?

It looks great! Thank you so much for your work on this, @Ckoerner!

I have two very minor suggestions, but I think it is good to go regardless, even without any changes. For the "how often do you upgrade" question, I wanted to answer in a less time constrained manner: "every time there is a point release", "every time there is a security-related point release" or "every time there is a major release". And, would it be possible to move the links from the page with the Submit button (where it is tempting to click them and lose the survey results) to the Thank You page?

@RHeigl Thanks for the OK and the correction.

@cicalese I can move the links to the thank you page. Good idea.

As for the upgrade question, you have an interesting point. The release schedule for MediaWiki is major releases every 6 months and point releases monthly.

https://www.mediawiki.org/wiki/Version_lifecycle#Release_policy

Which is more important to know? That folks update to major/point releases or that they're updating with any sort of frequency? Do you think we should combine the two answer types?

Ckoerner writes:

Which is more important to know. That folks update to major/point
releases or that they're updating with any sort of frequency? Do you
think we should combine the two answer types?

No comment on either of those points, but I would point out that there
have been questions on mediawiki-l about the lack of point releases.

Yes, that was why I was mentioning it. It was difficult to answer the question with a strict timeframe. We update whenever there is a release available to update to, but that does not appear to happen on schedule at this point.

I'm stuck on the 'How frequently do you update" question. Since the major/point release schedule isn't the most accurate, will responses tell us anything useful? Can we combine the two in a graceful way?

Sorry, I didn't mean for this to be a blocker. Despite the fact that the major/point release schedule is not currently accurate in terms of time, it is a driver for updating. I update when a new security-related point release or major release is available, regardless of whether that is every month or every six months. So, the time-based answers were not a good fit for my situation. But, some folks might use a more time based approach to updating. I think adding answers based upon release availability to the time based answers to the question would be a good solution, helping us to determine what the drivers are for updating. I don't recall if there was an "other" answer, but that could also help. But, I whatever you choose, this should not stop the survey from moving forward.

Ok, we're at the Hackathon at Wikimania and talked about the question about how frequently you update.

We've reworded it to ask, "Which MediaWiki update cycle is closest to what you use?"

We (Mark, Marcus, and myself) are at the hackathon in Mexico City. We've created a page to host a brief introduction to the survey (and eventually the results) on Mediawiki.org.

https://www.mediawiki.org/wiki/2015_MediaWiki_User_Survey

Great! Will you be posting information for remote participation in the Stakeholders' meetup?

The survey has been sent to mediawiki-l

http://hexm.de/MWSurvey

Alternate link: https://docs.google.com/forms/d/1j2ThONROvc0bGY5d3yx4tJln4Aa2Fs130j5OlHbjW14/viewform

Please share and provide suggestions on where else to promote. We will leave it open for a little more than 2 weeks, closing on Friday, July 31st.

Woo!

Ckoerner triaged this task as Medium priority.Jul 15 2015, 4:22 PM
Ckoerner updated the task description. (Show Details)
Ckoerner updated the task description. (Show Details)

Phew!
It's been a few weeks since Wikimania and I'm just now getting my bearings.

As of this morning we have 127 responses to our survey. That's a great accomplishment. Does anyone want to send out another reminder to the various mailing lists? One last hurrah?

I'll close the survey tomorrow (Friday) at the end of the day (20:00 CST).

Then we should put together a final report (maybe just an update of our presentation from Wikimania) with stats.

I can work on making the data available publicly - anonymized of course.

Then what?

Might I suggest we take the feedback from the survey, add it to our wishlist, and start to prioritize?

Exciting news and great job!