Page MenuHomePhabricator

Create abstraction for services to access database
Closed, InvalidPublic

Description

The recommendation API service needs to store pre-computed models in a database and expose them to API users upon request. Here's a patch for one use case. Another use case is related to porting the Python version of the API into the JS version linked above. Currently, the Python version needs to hold a large matrix in memory in order to compute a dot product with a vector. This is possible because of the availability of high quality scientific computing libraries such as numpy. Porting the algorithm as is to JS would be difficult because of the lack of decent libraries (although there seems to be a numpy like library for JS). Even then, we wouldn't want to port the algorithm as is because we can pre-compute the data and store it in a database. That's another use case where we need access to a database from a service.

Event Timeline

mobrovac subscribed.

As part of next FY's Platform Evolution CDP, we plan on working on and providing a re-usable key-value storage component that should cover the use case you are describing.

CommunityTechBot raised the priority of this task from High to Needs Triage.Jul 3 2018, 2:06 AM
Pchelolo subscribed.

The recommendation-api is now in production and is accessing MySQL. It's been decided in another ticket that key/value store would not be enough for the service as the queries needed could not be properly supported by KV. I think I'll close this one as invalid as the situation has changed since the task was filed and it's not quite relevant anymore.