Page MenuHomePhabricator

puppetize mysql m5-master access on parsoid-test server
Closed, ResolvedPublic

Description

the test server ruthenium
had a manually installed mysql server which is currently missing after a reinstall since it was not installed via the puppet roles

MySQL data has been imported to s5-master, grant puppetize credentials from the client server.

12:21 < ori> mysql db for parsoid testing -- running on bare metal in prod, but not user-facing, and if data is lost it's not the end of the world
12:22 < ori> where would you prefer it to live (on ruthenium or some other host) and how would you like it configured?


12:23 < subbu> it is a custom db with a fairly simple schema
12:24 < jynus> probably the best thing is to add a ticket with all background- plus requirements in space and load (+predictions of growth)
12:24 < subbu> it holds results from roundtrip tests ... about 300K rows created per test run (which can be a few itmes a week) .. so, over time, it builds up.

12:24 < ori> it's for diffing the output of parsoid and mediawiki

12:24 < subbu> no, also for diffing between test runs of parsoid ..
12:24 < jynus> so not mediawiki production
12:25 < jynus> misc, or something else
12:25 < ori> right
12:25 < subbu> we don't deploy parsoid before we verify tests there.
12:25 < jynus> internal service, basically
12:25 < subbu> visualeditor, content translation flow all depend on parsoid.
12:25 < ori> yep
12:25 < subbu> and mobile content services.
12:25 < subbu> yes, internal service.
12:26 < jynus> please add that (which I already understood), and give an estimation of size and load ,and its growth
12:26 < jynus> and I will give you some options

12:29 < subbu> jynus, i am creating the ticket now, but mutante copied over the entire database that used to reside on ruthenium (prior to reimaging it from ubuntu 
               12.04 to jessie) if you want to take a look independently.
12:29 < jynus> where?
12:29 < mutante> jynus: root@ruthenium:/mnt/data/mysql# 
12:31 < jynus> 145 GB

12:31 < jynus> that is considerable already
12:31 < subbu> jynus, https://github.com/wikimedia/mediawiki-services-parsoid-testreduce/blob/master/server/sql/create_everything.mysql is the schema

12:31 < subbu> we can purge old results periodically.
12:32 < subbu> and keep results from previous 3-6 months ... wihch will keep it contained.

Event Timeline

Dzahn created this task.Jan 25 2016, 8:41 PM
Dzahn assigned this task to ssastry.
Dzahn raised the priority of this task from to Normal.
Dzahn updated the task description. (Show Details)
Dzahn added a project: DBA.
Dzahn set Security to None.
Dzahn removed a project: Patch-For-Review.
Dzahn added subscribers: jcrespo, Dzahn, gerritbot and 5 others.

maybe duplicate of T124703 now

jcrespo renamed this task from (puppetize?) mysql setup on parsoid-test server to puppetize mysql m5-master access on parsoid-test server.Jan 26 2016, 6:12 PM
jcrespo updated the task description. (Show Details)
jcrespo updated the task description. (Show Details)
ssastry moved this task from Backlog to In Progress on the Parsoid board.Jan 26 2016, 9:35 PM
ssastry raised the priority of this task from Normal to High.Jan 26 2016, 9:45 PM

Change 266752 had a related patch set uploaded (by Subramanya Sastry):
ruthenium services: Add dummy secrets for parsoid-rt and parsoid-vd

https://gerrit.wikimedia.org/r/266752

Change 266753 had a related patch set uploaded (by Subramanya Sastry):
ruthenium services: Use puppetized db credentials for testreduce dbs

https://gerrit.wikimedia.org/r/266753

Yes, I am on this. Can you tell me what should the 2 accounts be for? Which grants? Which databases?

jcrespo claimed this task.Jan 27 2016, 3:35 PM
jcrespo moved this task from Triage to In progress on the DBA board.
  • user: testreduce
  • databases: testreduce_0715, testreduce_vd (although I wonder if it is simple to rename them to parsoid_rt and parsoid_vd .. it not, not to worry).
  • grants: as permissive rights as possible.

Creating the user now and puppetizing it.

Renaming a databases requires import and export to be safe, so we will skip that for now.

I need something else, host from which it will connect? Ruthenium?

Yes, ruthenium.

jcrespo added a comment.EditedJan 27 2016, 4:14 PM

Password will be on passwords::testreduce::mysql namespace, variable $db_pass, slightly different than your proposal on the "fake" puppet, hope that is ok.

Password will be on passwords::testreduce::mysql namespace, variable $db_pass, slightly different than your proposal on the "fake" puppet, hope that is ok.

Works for me. It is the same password on both databases, right?
Will update the two patches appropriately.

Works for me. It is the same password on both databases, right?

Well, in mysql databases or schemas (it is the same there, unlike other dbs), do not have passwords, but users do. I have created a single account to conntect from ruthenium with access to the 3 dbs.

Change 266752 merged by Jcrespo:
ruthenium services: Add testreduce::mysql password for db access

https://gerrit.wikimedia.org/r/266752

Change 266753 merged by Jcrespo:
ruthenium services: Use puppetized db credentials for testreduce dbs

https://gerrit.wikimedia.org/r/266753

jcrespo closed this task as Resolved.Jan 27 2016, 5:53 PM
jcrespo removed a project: Patch-For-Review.

Confirmed working at http://parsoid-tests.wikimedia.org/ with the new database host.