Hi @TAndic! We created this task to put together all the written content for the safety survey. Feel free to edit the task description above, thanks! We should also probably define a tentative deadline for this task as localization can take some time cc @Madalina.

Thanks @aminalhazwani ! Reviewing it today with @JAnstee_WMF to make some decisions and will update!

Hi @TAndic can you approve the strings in the task description above with a ✅ in the "Approved by GDI" column if we're good to go? Thank you 🙏🏼

I have only one small doubt regarding the "Thank you message text". Do you think is necessary to link to the project page after a user has already replied to the survey? My main rationale is that the content of the project page may be more useful if read before taking then survey instead of after. With the text we are probably targeting someone who experienced these issues and is willing to share/learn more about those specific issues, not the survey per se. What do you think?

Thanks @aminalhazwani ! Will update in a minute.

For the thank you message text, we are not necessarily targeting people to share/learn more about harassment issues in this specific instance, and are very much asking people to share more about the survey itself (if there are methodological/implementation/translation mistakes that they notice, if they want more information about why we're doing this and what the data will be used for, how to get the data about their communities once we publish it, if they want to be able to discuss with the Foundation/us implications of this research). If they have already responded to the survey but did not open the meta page from the question-portion, they may have no easy way to return to talk to us which can cause frustration on behalf of our community members/research participants. We need to be very explicit that people should not share their personal experiences on our talk page or other pages as this is also a risk to them and not within our research scope; they should instead follow the guidelines for reporting and addressing harassment through their specific community space, and would escalate to WMF if these issues are unable to be addressed through these routes. For an example of what forms of discussion we want directed towards us researchers/designers/QS folks and can anticipate, see the Talk Page of the 2015 Harassment Survey.

I have one question on the localization string:
From our conversation on auto-translation based on UI/language which can create issues with the "this" wording because one could potentially be confused about which Wikipedia language they are on if their browser auto-translates, do you think it would be better to think of these as separate tasks/questions? ie.

• For Commons, Wikidata, Meta, etc. the standard format can be used: "In the last 30 days, have you felt unsafe or uncomfortable contributing to $wgSiteName?"

For language-specific spaces (Wikipedia, but also would apply to Wiktionary, Wikinews etc), I think they may have to be separated:

• English: "In the last 30 days, have you felt unsafe or uncomfortable contributing to English $wgSiteName?"
• French: "In the last 30 days, have you felt unsafe or uncomfortable contributing to French $wgSiteName?"
• Spanish: "In the last 30 days, have you felt unsafe or uncomfortable contributing to Spanish $wgSiteName?"
• Portuguese: "In the last 30 days, have you felt unsafe or uncomfortable contributing to Portuguese $wgSiteName?"
• etc.

We need to be very explicit that people should not share their personal experiences on our talk page or other pages as this is also a risk to them and not within our research scope;

Big +1 on this @TAndic, and thank you for all the additional details. Taking all of this into account I believe we can keep the thank you message as it is.

Re: strings I am going to defer your question to @Jhernandez as he's much know knowledgeable than me. @Jhernandez, Tanja is proposing the following approach for making it clearer to users to which wiki community we're referring to when we ask them if they felt unsafe or uncomfortable. Is this something we can achieve with QuickSurvey?

Default initial message key:

• For Commons, Wikidata, Meta, etc. the standard format can be used: "In the last 30 days, have you felt unsafe or uncomfortable contributing to $wgSiteName?"

For language-specific spaces (Wikipedia, but also would apply to Wiktionary, Wikinews etc), I think they may have to be separated:

• English: "In the last 30 days, have you felt unsafe or uncomfortable contributing to English $wgSiteName?"
• French: "In the last 30 days, have you felt unsafe or uncomfortable contributing to French $wgSiteName?"
• Spanish: "In the last 30 days, have you felt unsafe or uncomfortable contributing to Spanish $wgSiteName?"
• Portuguese: "In the last 30 days, have you felt unsafe or uncomfortable contributing to Portuguese $wgSiteName?"
• etc.

In T294556#7492663, @aminalhazwani wrote:

Re: strings I am going to defer your question to @Jhernandez as he's much know knowledgeable than me. @Jhernandez, Tanja is proposing the following approach for making it clearer to users to which wiki community we're referring to when we ask them if they felt unsafe or uncomfortable. Is this something we can achieve with QuickSurvey?

Default initial message key:

• For Commons, Wikidata, Meta, etc. the standard format can be used: "In the last 30 days, have you felt unsafe or uncomfortable contributing to $wgSiteName?"

For language-specific spaces (Wikipedia, but also would apply to Wiktionary, Wikinews etc), I think they may have to be separated:

• English: "In the last 30 days, have you felt unsafe or uncomfortable contributing to English $wgSiteName?"
• French: "In the last 30 days, have you felt unsafe or uncomfortable contributing to French $wgSiteName?"
• Spanish: "In the last 30 days, have you felt unsafe or uncomfortable contributing to Spanish $wgSiteName?"
• Portuguese: "In the last 30 days, have you felt unsafe or uncomfortable contributing to Portuguese $wgSiteName?"
• etc.

It is doable but it will require creating a separate key per wiki which translatewiki translators will have to translate, and it will require configuring each survey differently when we deploy. In summary it would make the surveys harder to deploy and configure and making it harder for translators.

From our conversation on auto-translation based on UI/language which can create issues with the "this" wording because one could potentially be confused about which Wikipedia language they are on if their browser auto-translates, do you think it would be better to think of these as separate tasks/questions?

I'm not familiar with the conversation about auto-translation but wiki users can have their UI (and hence surveys) in their preferred language even if they are on a specific language wiki. For example, this is how I use enwiki https://en.wikipedia.org/?uselang=es, you can see the UI elements and the survey would be my language even if I am in English Wikipedia.

If google translate users are a concern then we should test how the question translates and see how much sense it makes but we should likely also try to know how many people do use it in case we are optimizing for something that is quite rare.

A couple of thoughts regarding making the project clear in the question: From my experience new editors don't really have a concept of english or spanish wikipedias, they just know Wikipedia, so clarifying the project language may not be so helpful to them. And on the other hand more experienced editors that may have acted across projects probably already know which wiki they are on, so the information might be kind of redundant.

A couple of thoughts regarding making the project clear in the question: From my experience new editors don't really have a concept of english or spanish wikipedias, they just know Wikipedia, so clarifying the project language may not be so helpful to them. And on the other hand more experienced editors that may have acted across projects probably already know which wiki they are on, so the information might be kind of redundant.

This is actually a solid point. Our initial concern originated from the first question "Do you feel safe contributing to this wiki community?" where "wiki community" might have sounded ambiguous. Since we are able to leverage the $wgSiteName variable this might be enough in terms of specificity.

@TAndic if you agree we could initially opt for the simpler approach which would only have one initial message key "In the last 30 days, have you felt unsafe or uncomfortable contributing to $wgSiteName?". This would enable us to simplify configuration, testing and deploy since we don't have to setup multiple instances of QuickSurvey.

Thanks @Jhernandez for the UI clarification, that makes much more sense!

Based on the different % coverage of each wiki, I had been working under the assumption that the configuration of the survey will need to be different for each wiki no matter what (see Phab Ticket T227793 for an example of how I've been assuming this would work; this one doesn't use translatewiki and the configuration is that the text begins with the language of the wiki -- English, Norwegian, and Arabic -- and the percent coverage is different for each one, .1, 1 and .5 respectively). I realize now that these are 3 separate configurations and three separate deployments rather than one and the same thing, which changes how I should thinking about this!

@Madalina and I had a discussion yesterday regarding whether translatewiki is necessary or if it would be better to configure separately, which we need to inquire into a bit more. My tendency is to err on the side of caution by naming the specific language space so we know our results are reliable for each space, but there might be other workarounds that could help us achieve specificity without having to deploy a different configuration for every wiki.

@aminalhazwani I think at the very least "this" Wikipedia or something similar to it would be necessary based on how I interpreted what @Jhernandez noted, regardless of translation:

And on the other hand more experienced editors that may have acted across projects probably already know which wiki they are on, so the information might be kind of redundant.

If we ask more experienced editors who contribute to multiple wikis if they have experienced feeling unsafe on Wikipedia (and not the specific Wikipedia they're on, or "this" Wikipedia), they may think we're asking them about Wikipedia broadly. This is how I would interpret the question as-is as someone who contributes on multiple language wikis. Aside from my own interpretation, my concern comes from a point a user brought up on the 2015 Harassment Survey discussion regarding separating global/local harassment measurement. This is precisely what we're trying to do, and as we're putting a lot of weight on only one question, I'm trying to be very cognizant of not creating any room for confusion and that the translations themselves are precise. Again, Commons, Wikidata, etc are exceptions as they don't exist as multiples.

Sorry if I'm seeming nitpicky (and more-so for adding to your workload), I just worry a lot could be at stake with this and don't want to make any quick decisions without exploring all of the options.

In T294556#7498367, @TAndic wrote:

Based on the different % coverage of each wiki, I had been working under the assumption that the configuration of the survey will need to be different for each wiki no matter what (see Phab Ticket T227793 for an example of how I've been assuming this would work;

Yes the config snippet will have to be separate per wiki because of the coverage, that assumption is correct! 👍 When I mentioned

In T294556#7495345, @Jhernandez wrote:

it will require configuring each survey differently when we deploy

I meant that each survey config will have both a different coverage and question key, needing extra care before deploying and more testing after to ensure all wikis got the question they should have.

this one doesn't use translatewiki and the configuration is that the text begins with the language of the wiki -- English, Norwegian, and Arabic -- and the percent coverage is different for each one, .1, 1 and .5 respectively). I realize now that these are 3 separate configurations and three separate deployments rather than one and the same thing, which changes how I should thinking about this!

I see why we were thinking differently about this seeing how that other survey was deployed. We were thinking about having a set of messages in the WikimediaMessages extension, and using TranslateWiki to input the translations, and using those keys for configuring the surveys for the deployment.

In that other survey, they opted for inputting the messages manually on each wiki using the interface messages in the Mediawiki namespace. These approaches have different tradeoffs, and we can opt for any of them:

In any case as you can see there are some tradeoffs, if you are making a survey for just a few wikis, then making the messages can be feasible, but may limit users with a different interface language than the one posted in the wiki. If you are making a survey for many wikis, for example 20, you will have to create 20 * 5 (maybe more if there is a custom thanks and extra info messages) pages, so ~100 pages with the messages in 20 wikis where you need to have administrator or interface rights to do so. Not the most straightforward method.

If we configure the survey with the messages in code and they go to translate wiki, they can be edited in an interface in translatewiki and translated to many languages in more straightforward ways and if you view the survey in enwiki but your language is Spanish for example, you will be able to answer it. I'm not sure how common this case is though.

There may be room for a mixed approach, where we upload the keys to WikimediaMessages, and create specific overrides per wiki in the MediaWiki namespace for the question, where it makes sense. Mediawiki namespace messages override the TranslateWiki ones if present.

Separately is the matter of how many deployments we make, that will depend on timeline and needs. Ideally from the engineering side we would deploy all surveys for the same survey at the same time and remove them at the same time but specially for turning them off it is understandable if some need to be turned off earlier than others due to different response rates and coverage.

@Madalina and I had a discussion yesterday regarding whether translatewiki is necessary or if it would be better to configure separately, which we need to inquire into a bit more. My tendency is to err on the side of caution by naming the specific language space so we know our results are reliable for each space, but there might be other workarounds that could help us achieve specificity without having to deploy a different configuration for every wiki.

Definitely needs to be thought out. I hope my comments above will help.

And on the other hand more experienced editors that may have acted across projects probably already know which wiki they are on, so the information might be kind of redundant.

If we ask more experienced editors who contribute to multiple wikis if they have experienced feeling unsafe on Wikipedia (and not the specific Wikipedia they're on, or "this" Wikipedia), they may think we're asking them about Wikipedia broadly.

Very good point, there needs to be some specificity about what the question is asking. The question is if this can be asked in a generic way. Something like "Do you feel safe contributing to this community" would be ideal in terms of simplicity to deploy :D

The question is if this can be asked in a generic way. Something like "Do you feel safe contributing to this community" would be ideal in terms of simplicity to deploy :D

I believe we've made it back full circle (re: "this wiki community")! 😂

Is there something like $wgSiteName that can generate the shorter URL, eg. "fr.wikipedia.org" "pt.wikipedia.org" etc.?

We could then do a variation on,

"In the last 30 days, have you felt unsafe or uncomfortable contributing to $wgSiteName (fr.wikipedia.org)?"

If it helps us decide, for these surveys we will start with 5 spaces, and maybe move up to 10 in the FY (but the more complex solution that seems simple now could snowball...)

These are the magic words that are available: https://www.mediawiki.org/wiki/Help:Magic_words

Maybe {{SERVERNAME}}?

• Enwiki: en.wikipedia.org
• Commons: commons.wikimedia.org
• Mediawiki: www.mediawiki.org
• Wikidata: www.wikidata.org

I've had a chat with Amir on slack and they mentioned that we should always strive for having messages in TranslateWiki when possible, because the on-wiki messages are not translatable, need to be duplicated by wiki, and are harder to clean up after the surveys are done. Of course there may be exceptions to this general rule, just some info from their perspective.

The Mediawiki: messages method is intended for overriding strings coming from translatewiki for specific wiki communities / purposes.

@aminalhazwani what do you think about

"In the last 30 days, have you felt unsafe or uncomfortable contributing to $wgSiteName ({{SERVERNAME}})?"

Anything I could be missing with this from your perspective?

Points noted from Amir, thanks for the further consultations! I think in general this makes total sense, hopefully it will apply to us as well. If SERVERNAME works I that should resolve my hesitations with translatewiki.

I agree on trying as much as possible to go through TranslateWiki. The manual input in the MediaWiki namespace method doesn't seem to be not only an issue of scope or complexity of deployment but might also create larger long-term issues. Also it looks like we would be locking out people from being able to edit and translate to other languages which feels a bit uncool to me.

SERVERNAME might be a good compromise if it works as intended.

I made a quick mock to visualize @TAndic proposal. This is the smallest scenario on a 320px wide screen.

With a combination of Magic Words and Parsers Functions we are able to dynamically change the question based on the {{SITENAME}}. The following code

In the last 30 days, have you felt unsafe or uncomfortable contributing to {{SITENAME}}{{#ifeq: Wikepedia | {{SITENAME}} | <nowiki> </nowiki>({{SERVERNAME}})}}?

outputs the following message on en.wikipedia.org

In the last 30 days, have you felt unsafe or uncomfortable contributing to Wikipedia (en.wikipedia.org)?

but nfortunately this method isn't working for Wikipedias that set their site name in a non-latin alphabet. For example @TAndic tested this on Serbian Wikipedia and the output does not include the {{SERVERNAME}} as intended

In the last 30 days, have you felt unsafe or uncomfortable contributing to Википедија?

I am going to open a SPIKE task to investigate this with engineering support.

To specify, not just non-latin alphabet but anything that doesn't exactly output "Wikipedia"; it is the same for Portuguese, ie. Wikipédia, and will likely also result in that if the grammatical case of "Wikipedia" changes for the sentence, ie. for Polish it may be "Wikipedii"

Edit: Updated T295751 to reflect this!

Hello hello! Some updates on this task! Joaquin was able to find a solution to dynamically change the question based on the hosting project. We would have to define all the specific projects for which we want to hide the URL, like this:

{{SITENAME}}{{#switch: {{SITENAME}} | Wikimedia Commons | Wikidata | Meta = | #default = <nowiki> </nowiki>({{SERVERNAME}})}}

I presented some updates on the Safety Survey to the broader design team and I collected some useful feedback that I would like to share with you all.

While the specificity provided by the full URL is useful it may be too technical for some users and the parenthesis may be distracting while reading. We were wondering if there's space to make the question lean more towards natural language. For example something like "In the last 30 days, have you felt unsafe or uncomfortable contributing to English Wikipedia?"

Pau and Rita provided some useful links to existing work on Translatewiki. Pau shared this collection of localized project names, for example "English Wikipedia" is translated to “Wikipedia en inglés” in Spanish https://translatewiki.net/w/i.php?title=Special:Translations&message=MediaWiki%3AProject-localized-name-enwiki%2Fes. Rita shared the strings for the Growth team where they use {{SITENAME}}. It was interesting to see how volunteers moved the magic words around based on their needs, see this for example https://translatewiki.net/w/i.php?title=Special:Translations&message=MediaWiki%3AWelcomesurvey-sidebar-privacy-text%2Fcs.

With all things consider how do you feel like about a more natural written question?

Looks cool! Is there a way we could see this in action with our question?