Page MenuHomePhabricator

Newcomer tasks: evaluate link recommendation (Dec 2019)
Open, MediumPublic

Description

Right now, all the tasks we're giving newcomers come from maintenance templates, which say that an article needs some work in general.  We want to try to see if it would be possible to give newcomers something very specific to do -- like add a link to a specific word or phrase.

@DED from the Research team is trying out an algorithm that attempts to predict that a specific word or phrase should be an internal link. We want our ambassadors to give us some initial feedback on whether this algorithm is going in the right direction.

Here's what was done:

  1. Took five articles from each of Czech, Korean, Arabic, and Vietnamese Wikipedias that have an "add links" maintenance template.
  2. Put the wikitext from each article into a Google Doc.
  3. Highlighted in blue the text that the algorithm thinks should be a link.  This includes a score between 0 and 1 that says how confident the algorithm is.

Here are the tasks for the ambassadors:

  1. In the documents, leave a comment on each suggested link saying whether you think it should or should not be a link.  In other words, was the algorithm correct?  You can do this by highlighting the blue text and leaving a comment.
  2. Leave comments on a few words or phrases (maybe 5) that you think the algorithm should have suggested as links, but did not.  For instance, in the first Czech article on Milton Friedman, the algorithm probably should have suggested that "ekonom" be a link.

A couple things to note:

  • @DED mentioned that the algorithm is currently guessing conservatively, and is therefore likely missing many links that it should suggest.  He plans to improve that in the future.
  • The algorithm doesn't account for only recommending the first occurrence of a word or phrase.  That can be fixed in the future.

Here are the documents for each wiki:

Details

Due Date
Jan 6 2020, 8:00 AM

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 23 2019, 10:39 PM
MMiller_WMF set Due Date to Jan 6 2020, 8:00 AM.Dec 23 2019, 10:45 PM
MMiller_WMF updated the task description. (Show Details)
Dyolf77_WMF moved this task from Backlog to In progress on the User-Dyolf77 board.
PPham added a comment.Dec 27 2019, 2:42 PM

I'm done. The funny thing is I can't think of any phrases or words that should be linked but isn't. I guess it's because those articles are too short, there's not enough option. And somehow out of the 5 articles of Vietnamese, the Algorithm (or somebody) picked one having only 1 sentence...

also done, Same thoughts as @PPham, too short articles! Overall, a quite interesting suggestions, it would be more efficient in longer articles.

☑. Those I did not comment on was good link.

Trizek-WMF triaged this task as Medium priority.Jan 6 2020, 3:08 PM
Trizek-WMF updated the task description. (Show Details)Jan 6 2020, 3:11 PM

@Urbanecm, any update on your side?

@Urbanecm, any update on your side?

I'm sorry, late on this one - I'll do my best to finish it today.

Trizek-WMF updated the task description. (Show Details)
Dyolf77_WMF moved this task from In review to Closed on the User-Dyolf77 board.Jan 18 2020, 8:39 AM
MMiller_WMF renamed this task from Newcomer tasks: evaluate link recommendation to Newcomer tasks: evaluate link recommendation (Dec 2019).Sat, Feb 15, 2:00 AM