This task is for tracking deploying the Recommendation API to production.
https://etherpad.wikimedia.org/p/recommendation-api-productisation
Production pipeline is described here: https://meta.wikimedia.org/wiki/Research:Technology/Article-Recommendation-Pipeline-Overview
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| recommendation api: refactor profile, remove module | operations/puppet | production | +13 -34 |
The steps to follow for deploying are documented here: https://wikitech.wikimedia.org/wiki/Services/FirstDeployment
I'm going to go through this and create the appropriate subtasks for tracking visibility
@mobrovac I've captured the steps of https://wikitech.wikimedia.org/wiki/Services/FirstDeployment in subtasks of T165760; is there anything else that should be captured in productionizing the service?
@akosiaris says that unless something comes up after @Joe reviews it, beta should be the last step before landing into production
It is my understanding that the subtasks as they exist now capture the scope of work to get the service into the beta cluster (from Research's perspective).
@akosiaris @Joe @mobrovac Could you please confirm for @leila?
@schana aside from a potential task for getting the service running beta, I think the tasks above are OK.
FYI: the service is active in BetaCluster, it resides on deployment-sca0[1-4] (n the standard deployment location) and is up on port 9632. @Joe @akosiaris please proceed with the review.
I have created via https://horizon.wikimedia.org/project/proxy/ a Web proxy for:
| hostname | backend | instance |
|---|---|---|
| recommendation-api-beta.wmflabs.org | http://10.68.20.183:9632 | deployment-sca01 |
That does not seem to work though (nothing reach the instance), I don't even know whether the other proxies are working.
Workaround for now:
$ ssh -N -L 9632:127.0.0.1:9632 deployment-sca01.deployment-prep.eqiad.wmflabsAnd in another terminal:
$ curl http://127.0.0.1:9632/That is because we have no more security groups available in the deployment-prep project, so that port cannot be exposed to the outside world. I will try to fiddle with the existing groups and get it in.
@Joe and/or @akosiaris, could you give a rough timeline of when you'll be able to get to this? Thanks
There is a quota per project. That can surely be bumped by cloud team. Anyway, deployment-sca02 and deployment-sca04 have the sca security group applied and rules can simply be added to it :]
Mentioned in SAL (#wikimedia-releng) [2017-07-07T14:53:23Z] <hashar> deployment-prep: add port 9632 to security group "sca" https://horizon.wikimedia.org/project/access_and_security/security_groups/593/ - T148129
Mentioned in SAL (#wikimedia-releng) [2017-07-07T14:53:58Z] <hashar> deployment-prep: change webproxy http://recommendation-api-beta.wmflabs.org/ to deployment-sca02 (has the proper security rule) - T148129
@schana http://recommendation-api-beta.wmflabs.org/ should now be reachable from anywhere and it points to deployment-sca02. Enjoy!
@schana I will get into the details early next week - I just have one additional question: who will be the consumer of the service exactly? The user browser? mediawiki via an extension? another software?
Change 364221 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] recommendation api: refactor profile, remove module
ContentTranslation, the user browser (via GapFinder), and any other tool that wants to consume it (nothing at this time)
To add to schana's list, I know Tofawiki gadget uses the API as well, and there may be more that we're not aware of.
Change 364221 merged by Giuseppe Lavagetto:
[operations/puppet@production] recommendation api: refactor profile, remove module