Page MenuHomePhabricator

Volunteer translator help for Community Insights Survey
Closed, ResolvedPublic

Description

What is the problem?

Qualtrics is translating the 2019 Community Insights Survey right now, and we have gotten respondent feedback in the past about the quality of the translation into languages other than English.

How can we help you?

Survey proofreading

We'd like to have the help of volunteer translators to help quality-check the translations before the survey goes live. Would someone on CommRel be able to help me get into contact with folks who can translate between English and the translated languages? The languages we're translating into are:

  • Brazilian Portuguese,
  • Italian,
  • Japanese,
  • Polish,
  • Russian,
  • Spanish (Latin America),
  • Ukrainian,
  • French,
  • German,
  • Chinese (Simplified),
  • Arabic,
  • Dutch.
Messages translated

4 messages have been drafted and have to be translated:

InvitationReminderReminderReminder
Brazilian Portuguese0%0%
Italian,
Japanese,
Polish,
Russian,
Spanish (Latin America),
Ukrainian,
French,
German,
Chinese (Simplified),40%
Arabic,
Dutch.

What does success look like?

Having a native speaker take a look at a portion of the survey in each of the translated languages to identify any issues.

What is your deadline?

  • 9 August, 2019 to get into contact with translators and set up a plan for a brief review. The Qualtrics translation will likely be ready to review between August 5 and 9
  • Feedback has to be sent on August 23 (to be confirmed)
  • The survey needs to go live on 1 September.

Event Timeline

Rmaung created this task.Jul 19 2019, 9:02 PM
Restricted Application added subscribers: Base, Aklapper. · View Herald TranscriptJul 19 2019, 9:02 PM

Asking out of personal curiosity (I am not in CommRels): Is "we" WMF's L&E? Or someone else? (Asking as your user page is unfortunately completely empty). I'm also curious why there is another review, as a complete review already took place in early June?

We does refer to L&E, this is my user page but I have not been able to get a MediaWiki account linked to that global login (I have the same email that I did when I set this one up as an intern). I'm trying to figure it out, sorry for any confusion.

Any review done so far has been of the English version of the survey. This is a request for help to review the survey translations into twelve other languages, as we have had issues with translation quality in the past and we don't know the source of those issues. That translation is being done by a third party (Qualtrics) and has not yet been completed. This request is for help assessing the quality of the 12 other language versions of Community Insights.

Aklapper added a comment.EditedJul 20 2019, 12:31 AM

Ah, thanks for the clarification! (You could edit the empty user page of the account and link it to the non-empty user page of your other account.)

Any review done so far has been of the English version of the survey.

That's not correct. Reviews of Qualtrics' translations took place between May 24th and June 16th (at least for some languages).

Thanks for the tip--I'll update that user page!

Those reviews were actually of the 2018 survey, and it has been significantly revised since last year to align with MTP goals and take less time to complete. The goal of that May/June review was to assess whether we had systematic translation problems before we had Qualtrics translate this year's survey. That translation (for 2019) is still in progress by Qualtrics.

Ah! Alright, and I apologize for any confusion! (Wanted to make sure that there is no misunderstanding that might lead to duplicated work and efforts.)

No worries, I appreciate it!

Elitre added a subscriber: Elitre.Jul 22 2019, 1:25 PM

Thanks, triaging

Trizek-WMF added a subscriber: Trizek-WMF.

I'll assist you the best way I can, @Rmaung.

Trizek-WMF triaged this task as Medium priority.Jul 22 2019, 3:02 PM

@Rmaung, are we going to use the spreadsheet to get the translations done?

We need to have a straightforward workflow for volunteer translators. Something like the source and the translation close to each other. Access a first document, and then comment elsewhere is not the most convenient way to have it done.

At the moment I see in the spreadsheet that the following languages have been covered: Spanish, German, Japanese, French, Italian, Chinese. Are those languages okay? Do you need people to confirm the reviews made by staff about those languages?

I've scheduled a meeting to discuss about all of this! :)

Trizek-WMF raised the priority of this task from Medium to High.Jul 25 2019, 10:21 AM

@Trizek-WMF, I agree that we need a workflow! I'm working on a similar spreadsheet for the 2019 survey translations to make these processes clear.

The spreadsheet you've linked above, which I believe Jaime Anstee put together, was a translation check on the 2018 survey translations before we proceeded to have the survey administration company translate the 2019 survey, This Phabricator ticket is intended for help with a quality check on the 2019 survey (much-changed from 2018!) and its translations. Those translations are in process right now, so I will need to wait for Qualtrics to finish before I can fully populate the new spreadsheet. What this ultimately means is that even for languages that were checked in May, we are asking for another review.

I look forward to discussing this with you on Friday!

Notes from the meeting:

Translations are done by Qualtrics. The goal is to have a quality check.

There is 158 items to review, which include

  • 1 high priority item with a gender issue,
  • 38 of medium priority.

Fixes and feedback would be added to Qualtrics August 23rd. To be confirmed. The best is to have two weeks of review by community members. We need to clarify how the volunteers will be contacted, but the overall idea is to send them some instructions and regularly check if the work is done.

At the moment, the process is to open Qualtrucs and a spreadsheet, and to switch from one to the other. The best way to have the work done it to have everything for work at the same place. @Rmaung will import the translated items to the spreadsheet so that people can look at the translations and make their observations at the same time. A link to test the survey will be provided, so that users will be aware of the context.

I'm on vacation after Wikimania (from August 19 to August 28), which means that I'll not be able to cover the end of the project. I will find someone to be my backup.

Trizek-WMF updated the task description. (Show Details)Jul 26 2019, 3:32 PM

@Rmaung, do you have a possible due date for the export of Qualtrics translations in a spreadsheet?

From the spreadsheet as it exists now for last year's reviews, I think we should move from:

Item order of appearanceQuestion ID #PriorityReviewerReview StatusAny issuesProblem description

to something that requires less actions:

Item order of appearanceQuestion ID #PriorityTextReviewerOK or Suggestion

The 4 fists columns will be populated by you, and the volunteers would only have to fill the 3 last.

  • reviewer: adding their name manually (if we really need it?)
  • OK or Suggestion, with two possible status:
    • ok (fills in green)
    • suggestion, because there is a change to make

Wording if, of course, to be improved.

What do you think?

@Trizek-WMF I like that format, I'll update the sheet!

We do not yet have a date for the Qualtrics translation export. There is apparently an issue with the contract and translation has not yet begun. We are actively looking into this, but unfortunately do not yet have a clear answer about when the translated items will be made available. I apologize for the delay and unclear timeline-- I will update as soon as we know more!

@Trizek-WMF We have confirmation that translation has been started by Qualtrics, and they anticipate that it should be complete by early next week (Hurray!). That will give me enough time to move all the translated items into the spreadsheet by 9 August (Friday of next week), which will keep us on the timeline we were hoping for. Let me know if this timeline still makes sense on your side!

9 August-- Spreadsheet becomes available to volunteers
9 August - 23 August-- Translation quality checks in progress

That will give L&E a week to resolve any translation issues before the survey goes live on 1 September.

Thank you for the update.

In the meantime, we should take the opportunity to work on the 4 invitation messages that we will send to community members. María already asked my team to assist you and it is natural addition to this task.

The process would be:

  1. import the messages on meta
  2. mark them for translations
  3. invite translators to work on them

It would be a good way to get them started.

mcruzWMF added a parent task: Restricted Task.Aug 1 2019, 4:42 PM

@Rmaung, any update concerning the invitations? We can start working on it soon, to spare time.

Trizek-WMF updated the task description. (Show Details)Aug 7 2019, 12:51 PM

I've added the need of translations for the following messages to the task description.

  1. Invitation
  2. Reminder
  3. Reminder
  4. Reminder

None of them are ready for now, since I have left a few questions for the CI team in their coordination document. When the questions will be cleared, I will start the translation process.

@Trizek-WMF I will take a look at the comms doc this week.

We are still on schedule for the translations to be ready for review on Friday. Qualtrics has finished all but one language, and I'll have the first three into the spreadsheet today, all 12 by Friday.

@Rmaung sent the email about the main document to the translators: https://lists.wikimedia.org/pipermail/translators-l/2019-August/004947.html

I will do the reminders and follow-ups.

I've pinged the translators who volunteered last year (only the ones whose language is a target one for this year).

So far, we have reviewers for Ukrainian, Arabic, German, Japanese, Spanish, Russian and French.

Trizek-WMF updated the task description. (Show Details)Aug 18 2019, 10:39 AM
Trizek-WMF added a comment.EditedAug 18 2019, 1:37 PM

We have a Polish translator.

French is done, and I asked the user about how long time it took him to finish the proofreading. Here is a translation of his reply, that includes sme very useful feedback for the 2020 survey:

It must be in the history but roughly 7am by going methodically. The problem is that the xls is on google doc (long a load), it is very large (pb of the displacement of a xls under browser despite widescreen HP 1680x1050), the columns are protected (I can not hide them or the reduce), the elements to be compared are not aligned, the EN / FR columns are not side by side, we repeat its name in front of each line (whereas it is me connected google which reads, and we repeat ' completed 'in addition to the validation' Issue Yes / No 'otherwise you will have to see the look of the translated form with these elements.

I'm off now. @Whatamidoing-WMF volunteered to work on this task while I'm off.

Page for translations coordination is now https://meta.wikimedia.org/wiki/Community_Insights/Translation .
@Trizek-WMF , @Whatamidoing-WMF , now that Trizek is back, please discuss who can further help Rebecca with this task now, and sync with her on next steps. TYVM!

Trizek-WMF updated the task description. (Show Details)Sep 10 2019, 12:58 PM

@Rmaung, I've updated the task with:

What is the current status concerning the proofreading? Have you postponed the date? Should I ask for more efforts from the community?

@Trizek-WMF We had intended to keep the original deadline for the proofreading (August 23), but with the feedback we are getting about the un-reviewed translations (German, Polish, and Chinese seem to be particularly bad but I imagine I'll hear more about others) I do think we should keep the proofreading open and more actively ask translators with those language capacities for help. Some users with complaints about the translation quality had asked to help, so I will be sharing the review document with them as well. I can make changes as they come in while the survey is still live.

Some users with complaints about the translation quality had asked to help, so I will be sharing the review document with them as well.

Can you privately share those names with me?
Don't you have access to a list of names who offered to help last year? I have mine here. They have already been pinged twice.

@Trizek-WMF I just sent an email with the one user who offered to make corrections. I do have access to that list, yes!

Trizek-WMF closed this task as Resolved.Sep 18 2019, 4:43 PM

I just had a meeting with @Rmaung (and Johan). We have discussed about this task. The outcomes are:

  • The survey is live since a while. There are apparently no feedback about translations that have been proofread by volunteers
  • Next year process would start earlier and the survey would have more elements kept from this year.
  • Maybe translators would be hired from the community, to have quality translations from people who know what they translate about.
  • Rmaung takes care of the feedback she gets, and she is in touch with people who offered to help after spotting some issues.

My involvement is not needed anymore, hence I'm closing the task.