@bd808 I'm moving this to paused for now, as there aren't available resources to drive the task. For the future, if we still want to load the data into MySQL, it can be restructured to avoid the index limit.

In T170877#3513840, @mobrovac wrote:

+1. FWIW, I'm still waiting on @schana to respond before putting up a PR and changing the route officially.

In T170877#3506477, @GWicke wrote:

https://{domain}/api/rest_v1/data/recommendation/article/creation/{from_lang}{/seed_article}

@leila I think framing it as articles for creation would be confusing since it requires a pair of source and target languages. If a monolingual person is using it to find articles to create, this endpoint will be unable to help them. With respect to the "items" versus "articles," lists of articles are consistently returned under the "items" label throughout RESTBase. We should stay consistent with this established practice. The endpoint's documentation specifies that it is returning a list of articles.

The source parameter validation has been merged and deployed.

@DarTar the pull request has been updated if you wanted to weigh in on the wording: https://github.com/wikimedia/restbase/pull/849

@DarTar let me know if anything else needs done for the porting. It should now just be a matter of adding content.

I stripped away the design-specific portions of the site. It should now be ready to be loaded with content.

Repo forked here: https://github.com/wikimedia-research/landing-page

https://github.com/schana/recommendation-tool/pull/14

I've added an initial table with the different endpoints. Next up is changing from only having endpoints to linking to a more detailed page describing the specifics of that version.

@DarTar Is this site generated from markup somehow? Or is updating the content going to consist of editing HTML directly?

@DarTar I split off the documentation portion to T171739

I've migrated the issues from the GitHub repo to this task for more centralized tracking.

Moving this to GapFinder board.

In T171472#3474338, @Shyamal wrote:

Thanks, the pages load on the new code site but it appears that clicking on the "add translation" has no effect, it puts in the original text as it is. Not sure if I need to do some additional settings (but I certainly have not checked the "Don't use machine translation".

Filtering in GapFinder will need to be supported through the backend, otherwise there likely won't be enough results for GapFinder alone to perform the filtering. I've started working on what a backend query may look like in WDQS to include the relevant information for future filtering, but this will likely need amended since it doesn't include some of the information.

In T162912#3472710, @leila wrote:

@schana a few questions as I'm testing the improved API:

What is the logic to move away from the 6-month used in the original experiment to 1-month? Part of the reason for including more months was to be able to capture some notions of stability in views (versus articles that can receive high traffic only in a given month).

The main reason was to limit the computational requirements. This is by no means set in stone, but Joseph wanted to be careful about running the job over 6 months of data, so we'll need to coordinate with Analytics before running that job.

@DarTar that is currently the behavior of the alpha version.

Hi @Shyamal, thanks for reporting this. It looks like the title is getting double-encoded somewhere along the way. GapFinder is in the process of being migrated to a new codebase, and it doesn't look like this problem exists there: https://recommend-alpha.wmflabs.org/translation?source=ru&target=en

This has been implemented and is currently visible in the alpha version of the tool: https://recommend-alpha.wmflabs.org/translation

@DarTar These points have already been considered. Different types of recommendations will slot in to the path as https://{domain}/api/rest_v1/data/recommendation/{recommendation_type}/{params...}. Supporting algorithm versioning and whatever other metadata is relevant can be done by adding that data to the response. Swagger documentation will be present necessarily for integration into RESTBase; see the pull request or the service's spec for how that will look.

There's some related discussion in T147420, and it came down to this:

There's a pull request open here: https://github.com/wikimedia/restbase/pull/835

@bd808 Yes, it's a lot of indexing, but the table has records for every wikidata item and columns for every wiki, with the values being predictions that a particular item should exist in a given wiki. For performance reasons, sorting is necessary on every column. I could expand the data to be relational with tables per wiki and do joins, but that complicates the ingestion that will be somewhat frequent during the development of the algorithm.

@bd808 I created a database s53132__trex_p, but it seems that tables are limited to 64 indexes. Is there a way around this limitation besides exploding the data into its relational equivalent?

@mobrovac Do you know anything about this?

@mobrovac Is there anything outstanding on this task? It looks like some of the work was performed under T165760.

Update: the code has been ported to service template node and has been deployed.

The current schema looks like /{domain}/v1/translation/articles/{source}{/seed}.

@leila, here are some examples to compare:

• Apple
  • https://recommend-alpha.wmflabs.org/translation_test?source=en&target=de&seed=Apple
  • https://recommend-alpha.wmflabs.org/translation?source=en&target=de&seed=Apple
  • Observations (translation_test compared to translation)
    • Wider variety of recommendations
    • Recommendations that are not as similar to Apple are still important (ex: Hazelnut)
• Mona Lisa
  • https://recommend-alpha.wmflabs.org/translation_test?source=en&target=de&seed=Mona%20Lisa
  • https://recommend-alpha.wmflabs.org/translation?source=en&target=de&seed=Mona%20Lisa
  • Observations (translation_test compared to translation)
    • Again, wider variety whereas translation recommendations are a lot more focused on Mona Lisa itself
• Grand Prix motorcycle racing
  • https://recommend-alpha.wmflabs.org/translation_test?source=en&target=de&seed=Grand%20Prix%20motorcycle%20racing
  • https://recommend-alpha.wmflabs.org/translation?source=en&target=de&seed=Grand%20Prix%20motorcycle%20racing
  • Observations (translation_test compared to translation)
    • translation_test returns teams involved in the sport as well as other motorcycle sports, where translation largely results in specific races
• Rick and Morty
  • https://recommend-alpha.wmflabs.org/translation_test?source=en&target=de&seed=Rick%20and%20Morty
  • https://recommend-alpha.wmflabs.org/translation?source=en&target=de&seed=Rick%20and%20Morty
  • Observations (translation_test compared to translation)
    • translation_test didn't give any recommendations directly related to Rick and Morty, instead there were a variety of other TV show articles
    • translation primarily returned directly related recommendations

access requested: T170592