Page MenuHomePhabricator

Design schema for general Recommendation API
Closed, ResolvedPublic

Description

The current swagger spec only captures basic functionality for the specific use-case of article recommendations. We would like to have the API function in a way where it can handle arbitrary types of recommendations. This will require design work on the schema.

Details

Related Gerrit Patches:
research/recommendation-api : masterMove package into extendable types structure

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 31 2016, 6:40 PM
schana added a comment.EditedSep 1 2016, 1:00 PM

Here's something rough that I came up with:

How the paths look:

/types:
  {'types': [
      {'name': 'translation', 'version': 'v1', 'spec': '/translation/v1'},
      {'name': 'translation', 'version': 'v2', 'spec': '/translation/v2'},
      etc.
  ]}

/translation/v1/articles:
  {'articles': [
      {'title': ..., 'wikidata_id': ..., 'pageviews': ...},
      ...
  ]}

/{recommendation type}/{version}/whatever/is/specific/to/it:
  {defined within recommendation type}
  • should the base portion have its own version?

The python side of things:

change recommendation and api python packages to be namespace packages

python3-recommendation:

/recommendation/api
  /base
    - this would contain the base implementation for querying types
      that inspects what's available in types package
  /types {namespace package}

import pkgutil
pkgutil.walk_packages(recommendation.api.types.__path__, recommendation.api.types.__name__ + '.')
# https://docs.python.org/3.4/library/pkgutil.html#pkgutil.walk_packages

python3-recommendation-translation:

/recommendation/api/types
  /translation
    swagger.yml # references specific versions? not sure if this is possible
    v1.yml
    __init.py__ # would have some well-known entrypoint that the base
                  inspector would know how to access (or it could be in configuration)
    {other files to do whatever job is necessary}

After thinking a bit, it seems like it would be more simple to not have the namespace-mangling. We could use the flask-extension pattern of having different types of recommendations living in recommendation_{type name} packages and just put it all together in the .wsgi file.

schana updated the task description. (Show Details)Sep 5 2016, 8:26 AM
leila added a comment.Sep 15 2016, 5:19 PM

@ori can you provide some feedback on this task? That will help us scope it and move forward with it.

schana triaged this task as Medium priority.Sep 20 2016, 8:22 AM
schana moved this task from Backlog to In Progress on the Recommendation-API board.
ori claimed this task.Sep 26 2016, 3:54 PM

@ori I've created a sample implementation following the pattern above at https://github.com/schana/swagger-based-api. Could you look at it when you have time and provide feedback here?

Change 316557 had a related patch set uploaded (by Nschaaf):
Move package into extendable types structure

https://gerrit.wikimedia.org/r/316557

Another thing I think would be good for the API to support is some form of pagination (T124499).

Here's a proposal that would involve adding a not parameter:

  1. First query
    • GET /types/translation/v1/articles?source=en&target=de
    • Response: [{/* article */}, ...]
  2. If more are desired
    • GET /types/translation/v1/articles?source=en&target=de&not=Some+Article&not=Some+Other+Article

Pros:

  • Doesn't require any session handling
  • Simple to implement

Cons:

  • Clients have more to manage

Change 316557 merged by jenkins-bot:
Move package into extendable types structure

https://gerrit.wikimedia.org/r/316557

schana closed this task as Resolved.Dec 12 2016, 7:41 PM
schana claimed this task.
schana removed a project: Patch-For-Review.
schana moved this task from In Progress to Done on the Recommendation-API board.