Now that the link recommendation algorithm has been productized into an API, we want to re-evaluate its results to ensure that it is still performing at a high level and that no regressions occurred as it was productized. We are expecting around 75% of link recommendations to be accurate.
Here's the procedure:
# Make sure that you have **[[ https://chrome.google.com/webstore/detail/json-formatter/bcjindcccaagfpapjjmafapmmgkkhgoa | this browser extension ]]** installed in Chrome, which will allow you to easily read the results from the API.
# @Urbanecm will list 30 random articles from fr, en, vi, ar, cs, and bn in **[[ https://docs.google.com/spreadsheets/d/1RH5mMC1oTwc_DxE-R6ToEv1cver4Aoh21Ur9XuYIG9c/edit#gid=0 | this spreadsheet ]]**. They will be articles that have at least one link recommendation.
# For each article, open the article up and open up its API link to receive its recommendations. @Urbanecm_WMF will prepopulate the API links in the spreadsheet, but [[ https://api.wikimedia.org/wiki/API_reference/Service/Link_recommendation/Get_link_recommendations | this is the link to the API ]] for your reference.
# Look at each recommendation and decide whether you would choose "Yes" or "No". This judgment should be based both on whether that word or phrase should be a link //and// whether the link target is the right article to which it should be linked.
# In the spreadsheet, enter how many recommendations were given for the article and how many you would choose "Yes".
# Also enter whether the article is "short", "medium", or "long", according to your judgment.
# Add any notes or issues that you saw. Example of things to note are: clear reasons why the algorithm was wrong (e.g. "Suggests a link in the middle of a longer song title"), or if the algorithm recommends a link after that word's first occurrence in the article, or if it suggests linking to a disambiguation page.