Setup access from service to mysql
Closed, ResolvedPublic

Description

In T203039: Storage of data for recommendation API we determined that MySQL was a good fit for the recommendation API. A database has been created in T205294: Request to create database and account for recommendation API. Now we'd like to access this DB from the Recommendation API.

host to connect: m2-master.eqiad.wmnet
db: recommendationapi
TLS: disabled
user: recommendationapi and recommendationapiservice (for read only)
credentials: on private puppet

bmansurov moved this task from Staged to In Progress on the Research board.
mobrovac added subscribers: jcrespo, fgiunchedi, Joe, mobrovac.

The first step would be to add the user/pass combo to the private puppet repo (ping @Joe @fgiunchedi cna you help out for this step?). After that we can include it in the service's public puppet profile module (I can help with that). We also need to set up the appropriate firewall rules on the x2 hosts so to make them accessible from SCB (ping @jcrespo - let's coordinate this?).

mobrovac triaged this task as Normal priority.Sep 25 2018, 4:22 PM
jcrespo added a comment.EditedSep 25 2018, 4:28 PM

the user/pass combo to the private puppet repo

That is already done. Include in your puppet code:
include passwords::recommendationapi::mysql
and

$recommendationapi_pass        = $passwords::recommendationapi::mysql::recommendationapi_pass
$recommendationapiservice_pass = $passwords::recommendationapi::mysql::recommendationapiservice_pass

To setup your template config.

For the firewall, I need to know the source (mysql client) ips.

Friendly ping @Joe @fgiunchedi. Can you please help with this task? Thanks!

Friendly ping @Joe @fgiunchedi. Can you please help with this task? Thanks!

Looks like @jcrespo has been on this already and password is in place now!

Thanks @fgiunchedi.

@mobrovac no blockers left?

bmansurov updated the task description. (Show Details)Oct 15 2018, 1:44 PM

For the firewall, I need to know the source (mysql client) ips.

That's the SCB cluster (both eqiad and codfw).

@jcrespo has the same DB been set up in Beta as well? Can you share the credentials for that one as well?

Change 467661 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[operations/puppet@production] Recommendation API: Add MySQL connection config

https://gerrit.wikimedia.org/r/467661

Change 467664 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/recommendation-api/deploy@master] Config: Add MySQL connection info

https://gerrit.wikimedia.org/r/467664

Change 467661 merged by Filippo Giunchedi:
[operations/puppet@production] Recommendation API: Add MySQL connection config

https://gerrit.wikimedia.org/r/467661

Change 467665 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/recommendation-api@master] Add MySQL connection info

https://gerrit.wikimedia.org/r/467665

Change 467665 merged by Mobrovac:
[mediawiki/services/recommendation-api@master] Add MySQL connection info

https://gerrit.wikimedia.org/r/467665

Change 467664 merged by Mobrovac:
[mediawiki/services/recommendation-api/deploy@master] Config: Add MySQL connection info

https://gerrit.wikimedia.org/r/467664

Looks like this is done, @mobrovac?

Looks like this is done, @mobrovac?

We still need @jcrespo for these two things below (firewall rules and beta), so let's keep the ticket open until we figure them out.

For the firewall, I need to know the source (mysql client) ips.

That's the SCB cluster (both eqiad and codfw).

@jcrespo has the same DB been set up in Beta as well? Can you share the credentials for that one as well?

@jcrespo has the same DB been set up in Beta as well? Can you share the credentials for that one as well?

T207495: Puppet broken on deployment-sca01 has been opened for this issue as this is now causing puppet failure.

@jcrespo can you please help out with T205452#4674282? Thanks!

Sorry, there is an ops clinic duty to answer these kind of requests- I did my part which was creating the user account on production. I am not responsible for anything else- anybody can do an RC on beta repos, and I am definitely not in charge of those.

Sorry, there is an ops clinic duty to answer these kind of requests- I did my part which was creating the user account on production. I am not responsible for anything else- anybody can do an RC on beta repos, and I am definitely not in charge of those.

Created T207795: Create the recommendation api DB in Beta for the Beta side of things, but AFAIK we still cannot access m2-master from SCB.

mobrovac closed this task as Resolved.

It turns out SCB nodes already have access to the needed DB host. Resolving.