Page MenuHomePhabricator

App Editor Tasks language CAPTCHA API
Closed, DeclinedPublic

Description

This will be used to test whether or not a user is capable of translating into the destination language. There should be unlocked user levels for each destination language as well.

The CAPTCHA consists of a single multiple choice question. The question presents a description in the source language and three possible descriptions in the destination language. Two of the possible descriptions are from random articles, the third is the description from the presented source language description. The user unlocks the language by selecting the correct description.

GET language CAPTCHA endpoint with parameters for source and destination language codes returns something along the lines of

{ "id": "some_unique_identifier", 
  "article": {
    ...summary response here....
  },
  "options": [
    "Description A",
    "Description B",
    "Description C"
  ]
}

And then an endpoint where the client would POST something along the lines of

{ 
  "id": "some_unique_identifier", 
  "answer": "Description B"
}

Event Timeline

JoeWalsh created this task.Jan 16 2019, 5:25 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 16 2019, 5:25 PM

For posterity, I've already started discussing this at T212795#4884573.

JoeWalsh updated the task description. (Show Details)Jan 17 2019, 4:35 PM
Dbrant added a subscriber: Dbrant.Jan 17 2019, 4:55 PM

Besides the question of how to actually generate questions, I'm confused on this point:

There should be unlocked user levels for each destination as well.

Does this mean there are separate sets of "levels" for each language in addition to the user's contribution "levels"? Or that the contribution levels we've been discussing should be kept separately for each language?

Jhernandez triaged this task as High priority.Jan 17 2019, 6:37 PM
Jhernandez added a subscriber: Jhernandez.

Needs discussion at the apps engineering sync with RI to be fully fleshed out.

From initial discussions, this is another counter, just of language-test-from->to and the count will be either 0 or 1, as passed or not. More to be talked about soon.

bearND added a subscriber: bearND.Jan 17 2019, 6:41 PM

Maybe we could use randomInCategory, get a bunch of descriptions for the list of articles, and pick 3 most similar ones that are not exactly the same?

JoeWalsh updated the task description. (Show Details)Jan 23 2019, 5:37 PM
Tgr added a subscriber: Tgr.Jan 24 2019, 11:00 PM

We have discussed this in person, but just to have it in writing, this task needs a better rationale, both to evaluate whether it will be useful enough to be worth the engineering effort, and because that might impact implementation details. Some potential issues:

  • Choosing decent captchas is more complicated than just choosing words at random - for example, for items where the description is a proper noun, the translation will be more or less the same (but often not quite the same so impossible to filter out - accents might be different, name order might be swapped etc). (I am assuming here that the task really meant labels, not descriptions. With descriptions you get worse problems because there is little guarantee they are the same in different languages, and usually all articles in the same category have more or less identical descriptions.)
  • Depending on what kind of users we want to stop with the captcha, there is probably no point in a POST endpoint, as opposed to just identifying the correct answer in the initial response. On the other end of the spectrum, you might need some kind of throttling mechanism to stop users from just trying until they succeed by blind luck.
  • Being able to pick the correct translation of a word is not really indicative of being able to translate at an acceptable level.
Jhernandez lowered the priority of this task from High to Normal.Feb 13 2019, 4:52 PM
Jhernandez changed the task status from Open to Stalled.

This task may not be valid depending on what android finally decides they will be doing. Needs re-evaluating in a month or so.

Mholloway closed this task as Declined.Mar 13 2019, 7:53 PM

The language assessment task (T207684) was declined.