Page MenuHomePhabricator

article recommendation user testing recruitment messages and sampling
Closed, ResolvedPublic

Description

Create the message to use in mailing list and on-wiki recruitment, define which mailing lists and forums to recruit from.

Event Timeline

Capt_Swing claimed this task.
Capt_Swing raised the priority of this task from to Needs Triage.
Capt_Swing updated the task description. (Show Details)
Capt_Swing added subscribers: SBecker, gpaumier.

Draft message text:

The Design Research team at the Wikimedia Foundation is seeking volunteers to participate in a series of interviews about content translation on Wikipedia. The goal of the study is to understand how experienced translators find and select articles to translate from one Wikipedia to another. The results of this study will help improve automated article recommendation tools for content translators.

We would like to speak to editors who have experience translating articles across Wikipedia languages. The only requirements for participating in this study are
a) the editor must be available to speak with a WMF researcher over Google Hangout, and
b) the editor must have previously translated at least one Wikipedia article to or from English and one of the following languages: Catalan, Farsi, French, Italian, Spanish, or Swahili.

Each study session will be approximately 50 minutes in length, and can be scheduled at a time that works for all participants. Thank you for helping us improve content translation!

@Capt_Swing thanks for this. Few comments:

  • It's good to keep the emphasis of the message on creation and not on translation since the project is not about translation, though CX tool is used as a mean for it.
  • With the same token, we are really looking for bilingual people who can edit in the destination language and know the source language enough that they can get ideas from. Translators are one subset of that population. One thing we observed in the frwiki experiment was that quite a few editors took the recommendations and created them outside of the CX tool. We're not sure why: maybe because they'd prefer to create content from scratch, or because they like source editing, or they did it for some other reason.
  • It would be amazing to get some feedback from users who are willing to contribute sw -> en.
  • Is there a reason why you want to keep the source language en? The quality of recommendations can change when the source language changes so if we can afford it, I suggest having another source, too, either sw or es.

@leila

  • makes sense to me
  • also makes sense. I won't be testing the CX interface, just having people examine the recommendations. But I will ask them about whether they tend to translate (via CX or other method) vs. "get ideas from" the source to improve the target. In general, I've been using the term "translation" to cover a range of activities, from direct translation (either through CX or via other method) as well as paraphrasing, getting ideas for content to add, copying references, etc. But I see how that can be confusing.
  • I agree.
  • I don't! The message says "translated... TO or FROM English". Hearing about translating to English would be especially fascinating.

Thanks for the feedback!

  • I don't! The message says "translated... TO or FROM English". Hearing about translating to English would be especially fascinating.

my bad! thanks!

Capt_Swing renamed this task from article recommendation user testing recruitment messages and sources to article recommendation user testing recruitment messages and sampling.Oct 7 2015, 10:17 PM
Capt_Swing edited projects, added Design-Research; removed Design-Research-Archive.
Capt_Swing subscribed.

Gathered samples of recently active CX users in the languages currently supported by the recommendation tool. Based on Amir's sample queries here: https://www.mediawiki.org/wiki/Content_translation/analytics/queries#Number_of_users_who_created_pages

Users who translated at least 3 articles in the past month:

@SBecker will pull from these lists and post a version of the study invitation message on the user's page on the target wiki.