Page MenuHomePhabricator

Pre compute consine similarity of Wikidata items from navigation vectors
Closed, DeclinedPublic

Description

The Python version of the recommendation API uses navigation vectors to compute cosine similarity between Wikidata items. In order to do so, a big matrix (over a GB in size) needs to sit in memory and be used for computation. We can do better and pre-compute consine similarities and serve them from a database. This helps us get rid of the second instance of the recommendation API service and use the first instance for serving both translation and related articles based recommendations. We can later use the pre-computed data in the JS version of the recommendation API once T193746 is resolved.

Event Timeline

Vvjjkkii renamed this task from Pre compute consine similarity of Wikidata items from navigation vectors to fpdaaaaaaa.Jul 1 2018, 1:12 AM
Vvjjkkii triaged this task as High priority.
Vvjjkkii updated the task description. (Show Details)
CommunityTechBot raised the priority of this task from High to Needs Triage.Jul 3 2018, 2:06 AM