Build API to surface 'morelike' article recommendations for missing articles
Closed, ResolvedPublic

Description

Now that we have predictions for article translation for some language pairs, we could use these predictions for suggesting a 'morelike' API for missing articles. Given a wiki (source wiki), the user enters an article title that exists on the wiki. Our goal is to suggest similar articles that are missing from the wiki. Here's how it should work:

  1. Look up article's Wikidata ID;
  2. Look up article's title in other languages (for which we have predictions ready — see above link);
  3. Find similar articles in those languages using the 'morelike' API;
  4. Using the predictions for translation, filter out existing articles in the source wiki;
  5. Order articles by their predicted normalized ranks (normalized pageviews an article would receive in the next six months if created);
  6. Show these ordered article Wikidata IDs to the user.
  7. (Optional) We can also show our machine translated titles for these articles.

Related Objects

bmansurov triaged this task as High priority.
bmansurov moved this task from Staged to In Progress on the Research board.

Change 450601 had a related patch set uploaded (by Bmansurov; owner: Bmansurov):
[mediawiki/services/recommendation-api@master] Add translation based 'morelike' API for missing articles

https://gerrit.wikimedia.org/r/450601

Change 476024 had a related patch set uploaded (by Bmansurov; owner: Bmansurov):
[mediawiki/services/recommendation-api/deploy@master] Add configuration for 'morelike' API

https://gerrit.wikimedia.org/r/476024

Change 476024 merged by Mobrovac:
[mediawiki/services/recommendation-api/deploy@master] Add configuration for 'morelike' API

https://gerrit.wikimedia.org/r/476024

Change 450601 merged by Ppchelko:
[mediawiki/services/recommendation-api@master] Add translation based 'morelike' API for missing articles

https://gerrit.wikimedia.org/r/450601

The API has been built, we need to import data into MySQL for it to work, which is tracked in T208622: Import recommendations into production database.

Change 477674 had a related patch set uploaded (by Bmansurov; owner: Bmansurov):
[mediawiki/services/restbase@master] Add 'morelike' recommendation public API specification

https://gerrit.wikimedia.org/r/477674

Change 477674 abandoned by Bmansurov:
Add 'morelike' recommendation public API specification

Reason:
Github PR: https://github.com/wikimedia/restbase/pull/1084

https://gerrit.wikimedia.org/r/477674

Mentioned in SAL (#wikimedia-operations) [2018-12-06T22:58:45Z] <ppchelko@deploy1001> Started deploy [restbase/deploy@be8f0c0]: Add 'morelike' recommendation public API specification T201192

Mentioned in SAL (#wikimedia-operations) [2018-12-06T23:21:32Z] <ppchelko@deploy1001> Finished deploy [restbase/deploy@be8f0c0]: Add 'morelike' recommendation public API specification T201192 (duration: 22m 46s)

Mentioned in SAL (#wikimedia-operations) [2018-12-06T23:45:45Z] <ppchelko@deploy1001> Started deploy [recommendation-api/deploy@299b268]: Add 'morelike' article recommendations API T201192

Mentioned in SAL (#wikimedia-operations) [2018-12-06T23:47:51Z] <ppchelko@deploy1001> Finished deploy [recommendation-api/deploy@299b268]: Add 'morelike' article recommendations API T201192 (duration: 02m 06s)

bmansurov closed this task as Resolved.EditedFri, Dec 14, 3:13 PM
bmansurov moved this task from In Progress to Done (current quarter) on the Research board.

Fixes from T211423: Refactor and document the recommendation API are going live on Monday. New data import will happen today. Once live, the following Wikipedias will have this recommendation API:

  • en (sources: de)
  • es (sources: en)
  • fa (sources: en)
  • uz (sources: en, ru)

Old example URL: https://uz.wikipedia.org/api/rest_v1/data/recommendation/article/morelike/translation/Kitob

New example URL: https://uz.wikipedia.org/api/rest_v1/data/recommendation/article/creation/morelike/Kitob