Update week 2022-01-10:

• • onboarding collaborator (Akhil) to the current state of the project and upcoming tasks
  • sketched a protocol for evaluating the link recommendations to orphans: dataset, task+metric, baselines
• started to build a test-API to surface link recommendations for incoming links from link translation. in order to avoid scaling issues at this point, I am only yielding the top-3 choices and only for simplewiki.
  • the recommendation works for any article (not just orphans), e.g., for the article Monument we can query via https://linkrec.toolforge.org/api/v1/in?wiki_db=simplewiki&title=Monument which recommends to add the link from the article on Memorial, which already exists in 20 wikis (e.g. dewiki, enwiki, ptwiki, etc)
  • picking randomly an orphan article (from the category of all orphan articles) Boing (this article was already created in 2009 and still has no incoming links) we get recommendations via the query https://linkrec.toolforge.org/api/v1/in?wiki_db=simplewiki&title=Boing which yields several sensible suggestions from where to link this article (all of these link suggestions exist in other wikis)

Update week 2022-02-07:

• starting to build the evaluation dataset of links that were added to orphan-articles in one month. considering to change data-processing pipeline from pagelinks-table (containing any link in the article) to html-dumps which allows to separate links from, e.g., navigation templates which are not good candidates for recommendation.

Update week 2022-02-14:

• ongoing: building the dataset for evaluation of link recommendation to orphans. decided on an approach combining pagelinks-table with wikitext-dumps: while the latter contains only a subset of all links (see Mitrevski et al), it allows to keep track of which links appear in the text of an article (in contrast to links transcluded from templates which are less useful for recommendation).

Update week 2022-02-28:

• solving some details when building the evaluation dataset about which links to include from the text for recommendation (e.g. remove links from infoboxes, include links from lists such as "See also" sections) and how to parse wikitext accordingly.

Update week 2022-03-07:

• working to finish the dataset for the evaluation
• collecting and starting to implement baseline models to demonstrate that task is difficult; for example, the orphan-template suggests to use the Find link-tool which is based on a text-search looking for direct matches to the title of the orphan article in other articles. while the tool works good in many cases, for orphan-articles there are often no resutsl (example).

Update week 2022-03-28:

• evaluation of the link-recommendation is ongoing and will be completed during Q4
• we have generated a ground-truth dataset of all links that were added to orphan articles in all Wikipedias comparing two consecutive snapshots
• We are currently implementing the evaluation in terms of the recall@k:
  • given a true link (s_true,t) that was added to an orphan article as a target (t), our model generates a ranked list of the top-k links (s_1,t)...(s_k,t) to the orphan article as a target
  • averaging over all true links, we calculate how often the true link was among the top-k recommendations
• our link-translation model will rank the recommended links according to the number of language editions the link already exists

Update week 2022-05-02:

• finished first evaluation of link recommendation for orphan articles
• evaluation data: 68,473 orphan articles that were de-orphanized between two consecutive snapshots (2022-01 vs 2022-02), we extract the in-links that were added to these articles as ground-truth. Note that this is less than 1% of the total number of orphan articles that exist (>8M).
• we calculate recall@k, i.e. the fraction of times the added link was among the top-k suggestions from link-translation. Some example statistics for k=10:

• next step: compare with (simple) baseline from, e.g., recommendations from standard graph-embedding

Update week 2022-05-09:

• evaluating the baselines for comparison: i) simple heuristic based on reciprocating outgoing links; ii) nearest neighbors in embeddings of link network (currently training the embedding)

Update week 2022-05-16:

• investigated different heuristic approaches for baseline comparison. I ended up with 3 heuristics we can use almost out-of-the-box
  • link-reciprocity: suggest any of the existing outgoing links as new incoming link (can be obtained easily from the pagelinks-table)
  • morelike: recommend similar articles from cirrussearch' morelike (the top-3 suggestions are shown in the mobile version as "related articles"). can be easily obtained by querying the MediaWiki API
  • findlink-tool: this tool suggests articles to link from based on text-search. it is listed in the orphan-template as a way to find suggestions how to de-orphanize articles.

Update week 2022-06-27:

• figured out how to query morelike and findlink-recommendations as baselines
• started summarizing and writing up the results on the evaluation of the link recommendation for the meta-page

Update week 2022-07-04:

• running baseline evaluation
• cleaning up repo and writing results for meta-page (ongoing)

Update weel 2022-08-08:

• completed and compiled evaluation results of link recommendations for orphan articles (see spreadsheet)
• will close task when tables are moved to meta (hopefully next week)