Page MenuHomePhabricator

Community relations support for editor gender surveys
Closed, ResolvedPublic

Description

What is the problem?
The research team would like to run surveys that will measure gender diversity amount the editor population and would like to request support with communicating the upcoming changes and getting translations for the survey itself and related communications and conversations
How does success of this task look like? How do we know when we are done?

Is there any goal, program, project, team related with this request?
Research team
What is your expected timeline from start to end? Is there a hard deadline?
ASAP

Event Timeline

ovasileva removed Isaac as the assignee of this task.Jul 16 2019, 3:27 PM
ovasileva updated the task description. (Show Details)
Trizek-WMF triaged this task as High priority.
Trizek-WMF added a comment.EditedJul 17 2019, 3:41 PM

I'll assist you with this task. Some questions to start:

  • Which Norwegian are you targeting? Bokmål, Nynorsk or both? I haven't identified the language from the existing translations.
  • Where did you get the current translations?
  • How will they be displayed on wiki? Through custom mediawiki messages apparently, or something else?
  • Do you have an example of how the survey will look like on wiki?
Isaac added a comment.Jul 17 2019, 4:30 PM

Which Norwegian are you targeting? Bokmål, Nynorsk or both? I haven't identified the language from the existing translations.

Good question: Bokmål given that it's the larger community so better supports the sample size that we would need.

Where did you get the current translations?

Almost all of the existing translations come from a series of reader surveys we just ran (T212444). You can see the volunteers who helped on this meta talk page. Specifically, for Bokmål that was Jon Harald Søby and Arabic it was Abbad Diraneyya. They did an excellent job but they already provided a lot of support for those surveys recently so I did not want to continue to press them for support.

How will they be displayed on wiki? Through custom mediawiki messages apparently, or something else?
Do you have an example of how the survey will look like on wiki?

We are using the QuickSurveys extension, which will show a small survey box within an article that a logged-in user is reading. Here's an older mockup from mobile and a screenshot of how the current survey might look on desktop (on devwiki so ignore all the redlinks) is below. It does not display on main pages, outside the article namespace, in the mobile apps, or if DNT is turned on. The users can also dismiss it via Prefer not to say and that will be stored locally in the browser so that they no longer see the survey. The actual text of the survey questions/answers etc. are provided via MediaWiki interface pages -- example from the reader surveys (T226273#5279221) -- and these are pages that I currently have permissions to create and will do so once the translations are complete.

Trizek-WMF updated the task description. (Show Details)Jul 17 2019, 5:26 PM
Trizek-WMF updated the task description. (Show Details)

The users can also dismiss it via Prefer not to say and that will be stored locally in the browser so that they no longer see the survey.

You mean that it is the only option to dismiss it? Will it be displayed on every page until an action is taken? If so, could a "skip" button or a close icon be added?

I'll start to take care of the translations tomorrow, my time.

Isaac added a comment.Jul 17 2019, 6:12 PM

The users can also dismiss it via Prefer not to say and that will be stored locally in the browser so that they no longer see the survey.

You mean that it is the only option to dismiss it? Will it be displayed on every page until an action is taken? If so, could a "skip" button or a close icon be added?

Unfortunately yes this is the option for dismissing the survey. I asked the same question a while back (T225042#5280220) and adding the "Prefer not to say" was the compromise. Though on principle I still definitely think it's important to be able to dismiss these surveys, I'll say that when we run these for readers (which is a slightly different type of survey), the vast majority of people ignore them and relatively few dismiss them (there it's a "No" button for "Do you want to take a survey?"). I can't speak to the engineering support that would be required to make a more permanent solution. The other current (long-running) survey (example: https://en.wikipedia.org/wiki/Chicago?quicksurvey=true) is designed the same way but I do not know if they have received feedback about it.

I'll start to take care of the translations tomorrow, my time.

Thanks!! Just be aware of the title and paragraph associated with the privacy policies that also need translation. They are a bit buried so it's easy to miss them.

I've pinged translators. I had to change the sharing settings to allow them to edit the file.

In the task description, concerning "Obtaining translations for village pump conversations", what does it mean precisely? Do you need to get the translations of the message (included kin the request to translators) or do you need help to translate the replies? I may be lost in translations here. :)

Trizek-WMF updated the task description. (Show Details)Jul 18 2019, 1:23 PM
Isaac added a comment.Jul 18 2019, 2:15 PM

In the task description, concerning "Obtaining translations for village pump conversations", what does it mean precisely? Do you need to get the translations of the message (included kin the request to translators) or do you need help to translate the replies? I may be lost in translations here. :)

@Trizek-WMF : yeah, we're looking for translations of the message included in the translation docs (The Wikimedia Foundation [[mw:Wikimedia_Research|Research]] and ...) so that it can be posted in the editing community's language. for any replies, we might need support, but i suspect that Google Translate etc. will be good enough for getting the gist if the responses are in Norwegian / Arabic.

I've pinged translators. I had to change the sharing settings to allow them to edit the file.

No problem - thanks for taking care of that!

In the task description, concerning "Obtaining translations for village pump conversations", what does it mean precisely? Do you need to get the translations of the message (included kin the request to translators) or do you need help to translate the replies? I may be lost in translations here. :)

@Trizek-WMF : yeah, we're looking for translations of the message included in the translation docs (The Wikimedia Foundation [[mw:Wikimedia_Research|Research]] and ...) so that it can be posted in the editing community's language.

So it was what I understood. :)

for any replies, we might need support, but i suspect that Google Translate etc. will be good enough for getting the gist if the responses are in Norwegian / Arabic.

Yes, it works pretty well. We also have some ressources from those community to help us in case of a confusing translation.

Done for Arabic.

Isaac added a comment.Jul 19 2019, 1:46 PM

@Trizek-WMF thanks! I'll incorporate the translations and send the privacy policy over to be uploaded on the foundation wiki -- is there an account or individual who I can give credit to? For example, see the initial upload here for a related Arabic privacy policy: https://foundation.wikimedia.org/w/index.php?title=%D8%A7%D8%B3%D8%AA%D8%A8%D9%8A%D8%A7%D9%86_%D8%B9%D9%86_%D9%82%D9%8F%D8%B1%D9%8E%D9%91%D8%A7%D8%A1_%D9%88%D9%8A%D9%83%D9%8A%D8%A8%D9%8A%D8%AF%D9%8A%D8%A7&action=history

is there an account or individual who I can give credit to?

Yes. For Arabic, it is [[user:Haytham abulela]].

Trizek-WMF added a comment.EditedJul 23 2019, 2:46 PM

Bokmål is (almost) done.

@Isaac, what are the next moves?

Trizek-WMF updated the task description. (Show Details)Jul 23 2019, 2:49 PM
Isaac added a comment.Jul 24 2019, 2:12 PM

Thanks @Trizek-WMF ! I've confirmed that Bokmål is complete now too. At this point, I'm going to reach out to the Readers team (who will be supporting the deployment of this survey) and check whether they have availability next week. Do you think that would be sufficient time if we post these notices on the respective Village Pumps and Space in the next day?

I would ask that you stand-in as the point-of-contact in the Village Pump posts -- is that okay with you? I'll also be monitoring the talk page for our meta page for these surveys, which we link to in the posts.

As far as we warn the communities ideally 3 days prior to the deployment, it is fine. So we need to define a proper calendar.

I can post and monitor on the two village pumps pages.

Isaac added a comment.Jul 25 2019, 2:11 PM

Great -- I requested July 31st for the deployment and I think it's safe to say that's the day it will be deployed (perhaps +- 1 day but that still gives at least three days). So if that sounds good to you, let's fill in that date and move forward with posting. Thanks!

Trizek-WMF added a comment.EditedJul 25 2019, 4:15 PM

I'll post the messages on Monday then. Posting on Friday would be too early.

Isaac added a comment.Jul 26 2019, 1:26 PM

Sounds good to me -- thanks!

Trizek-WMF updated the task description. (Show Details)Jul 29 2019, 10:12 AM
Restricted Application added subscribers: jeblad, Danmichaelo. · View Herald TranscriptJul 29 2019, 10:12 AM

I've posted on both. We forgot to have a title translated, I reused the previous ones.

Isaac added a comment.Jul 29 2019, 1:49 PM

Thanks @Trizek-WMF ! Just an FYI that one of the pages I created (https://no.wikipedia.org/wiki/MediaWiki:Editor-gender-1-privacy), which contains the privacy policy link that is shown to survey takers in Bokmål, was nominated for deletion. It happened over the weekend so I just responded with more context and hopefully it will resolve itself, but we will need this page to run the survey. So if it is deleted, we will need to work with the community to resolve that.

The discussion is here: https://no.wikipedia.org/wiki/Wikipedia:Sletting/MediaWiki:Editor-gender-1-privacy

The deletion proposal has been closed as "keep" and awaits archiving. You've handled it a great way!

I've received two thanks for the message on Arabic Wikipedia. Which means that it has been read.

Isaac added a comment.Jul 29 2019, 2:40 PM

The deletion proposal has been closed as "keep" and awaits archiving. You've handled it a great way!

Ahh yes excellent news -- it was closed right after I posted that message :) Thanks!

I've received two thanks for the message on Arabic Wikipedia. Which means that it has been read.

Good to hear -- I'll continue to plan for Wednesday then unless I hear otherwise.

Nothing to report so far.

Isaac added a comment.Jul 31 2019, 4:28 PM

Thanks @Trizek-WMF : one issue came up during launch today. It turns out we're missing Bokmål translations for two buttons in the survey (sorry I wasn't aware until now):

The English versions are here:

So we'd need:

Could you help us with getting these translations? As soon as we have them, we can try launching again. Thanks!!

I've requested those translations on wiki.

Translations are supposed to go by the train. If you want to start earlier, you can override the local messages:
https://nb.wikipedia.org/wiki/MediaWiki:Ext-quicksurveys-internal-freeform-survey-no-answer-alert
https://nb.wikipedia.org/wiki/MediaWiki:Ext-quicksurveys-internal-freeform-survey-submit-button

Trizek-WMF added a comment.EditedAug 2 2019, 1:09 PM

Done for translations. I've imported them to Translatewiki.

  • «Please select an answer or enter your own.» → «Vennligst velg et svar eller skriv inn ditt eget.»
  • «Submit» → «Send inn»
Isaac added a comment.Aug 2 2019, 1:12 PM

Wow - thanks, quite quick! Much appreciated! These will hopefully go out Monday now

Isaac added a comment.Aug 2 2019, 1:19 PM

Translations are supposed to go by the train. If you want to start earlier, you can override the local messages:
https://nb.wikipedia.org/wiki/MediaWiki:Ext-quicksurveys-internal-freeform-survey-no-answer-alert
https://nb.wikipedia.org/wiki/MediaWiki:Ext-quicksurveys-internal-freeform-survey-submit-button

Also, will these local messages be automatically over-ridden now that the translation is up or should I go in and update them?

If you create local messages, they will have to be locally deleted if a global translation exists. Otherwise, the global translation is not taken into account.

Isaac added a comment.Aug 2 2019, 4:28 PM

Per conversation, I have updated the local pages with the translations and will monitor them just to make sure that when the official versions are pushed late next week, nothing breaks. See T227793#5388113 for links.

Nothing special to report from my side, community speaking.

Isaac added a comment.Aug 5 2019, 5:02 PM

Thanks - hopefully launching in one hour!

Trizek-WMF lowered the priority of this task from High to Normal.Aug 6 2019, 2:15 PM

I'm decreasing the priority of the task. It doesn't not mean that I'm leaving you. I'm just reflecting the amount of work I now have: monitoring the wikis is already in my routine, which was not the case when you asked me to find translators. :)

Trizek-WMF updated the task description. (Show Details)Aug 6 2019, 2:16 PM
Isaac added a comment.Aug 7 2019, 4:59 PM

Sounds good -- FYI I'm responding to some comments on the meta page as I see them: https://meta.wikimedia.org/wiki/Research_talk:Surveys_on_the_gender_of_editors

So far, it seems to go smoothly.

One piece of feedback I'm taking is that think of a preference that would allow users to be opted-out of any kind of survey would be a good idea.

Isaac added a comment.Aug 13 2019, 7:11 AM

Yes, I've been compiling the feedback and will make sure to include some recommendations regarding the functionality of QuickSurveys for this type of survey. In particular:

  • Ability to remove survey without responding explicitly. This should be a relatively straightforward UI fix but would require someone to do the work obviously.
  • Ability to opt out completely from surveys like this. This would likely be more complicated as it would have to be a change to the user settings, databases, etc. and we would have to determine if it only applied to QuickSurveys or other extensions as well.
  • Confusion around survey re-appearing in new browsers (or the same browser if cookies are deleted). This is how the sampling works but this is not at all evident to respondents, so there can be confusion when someone responds and then sees the survey again. I don't believe we can change the sampling strategy to be aware of whether a user account has already taken the survey in a different browser without compromising privacy, but we should consider having a better explanation of this within the survey to head off concerns for any future deployments.
  • General frustration/anger with the presence of the survey (in some part tied to the difficulty of opting out from it).

If you see other feedback I'm missing, please let me know. Thanks!

Elitre added a subscriber: Elitre.Sep 3 2019, 6:00 PM

@Isaac Is there anything missing here? :)

Isaac added a comment.Sep 3 2019, 6:33 PM

Thanks for the ping @Elitre -- I'm comfortable with wrapping up this task if @Trizek-WMF is. There shouldn't be any further community relations support until we start sharing out results, but we can open a new task then if it is deemed necessary.

Thanks for the ping @Elitre.

I've made a review of the wikis where it was posted, other than English Wikipedia @Isaac watched, and I have nothing to add to the list.

The concerns listed by Isaac are related to QuickSurveys, not to the survey content, except people who asked "why asking me since I selected my gender in my preferences?".

Feel free to mark the task as resolved. :)

Elitre added a comment.Sep 4 2019, 3:29 PM

Thanks both!

Elitre closed this task as Resolved.Sep 4 2019, 3:29 PM