Page MenuHomePhabricator

db servers for designate and labs pdns
Closed, ResolvedPublic

Description

Right now I'm setting up the designate labs service, which will use pdns with mysql as its backend.

Rather than set up local dbs for those two services, can I get a proper db set up on a db server?

setup info for pdns is here: https://doc.powerdns.com/md/authoritative/installation/#example-configuring-mysql

It's not obvious to me how the designate db should be set up since it's handled by a python script... I guess with a proper server url and account it can /still/ be handled by that script. http://designate.readthedocs.org/en/latest/getting-started.html

Event Timeline

Andrew claimed this task.
Andrew raised the priority of this task from to High.
Andrew updated the task description. (Show Details)
Andrew added a project: Cloud-Services.
Andrew added subscribers: Krenair, Springle, Andrew, Aklapper.

Note that both virt1000 and holmium (soon to be the designate server) are in a public vlan, not in labs-private. The stuff in labs-private talks to them via rabbitmq and db calls are marshalled through the queue as well.

Any idea what sort of load will this generate?

Any idea of the size of the eventual data set? (I'm guessing very small, but jic...)

Is the service visible/queryable to hosts outside WMF network, or entirely internal?

How does pdns relate to all the other databases supporting labs services on, eg, virt1000? Should we be choosing a physical production database server that will eventually handle pdns and [insert-other-stuff-here]?

After poking around I'm hoping the answers are: "small", "small", "no, internal only behind open stack", and "maybe we'll see".

So I've created the following:

pdns DB Server: m1-master.eqiad.wmnet
pdns DB name: pdns
pdns DB user: pdns
pdns DB pass: (see private repo passwords::pdns::$db_pass)

designate DB Server: m1-master.eqiad.wmnet
designate DB name: designate
designate DB user: designate
designate DB pass: (see private repo passwords::designate::$db_pass)

Both users have DML grants only; that is SELECT, INSERT, UPDATE, DELETE and CREATE TEMPORARY TABLE. They cannot perform any DDL (and I will complain if they must!).

If you're testing the python script please use the 'pdns' user only, not root or anything else similarly tempting ;-) If you must use root for setup at all, then please do test it somewhere else first :-)

I've gone ahead and created the tables for pdns as per[1]. I havn't created anything for designate except the empty database since a quick review of the docs didn't help.

[1] https://doc.powerdns.com/md/authoritative/installation/#example-configuring-mysql

Sorry for the slow response!

Any idea what sort of load will this generate?

Not much. Worst case is that pdns will hit the db every time a ttl expires. The other events will be very infrequent (e.g., one hit each time an instance is created or destroyed.)

Any idea of the size of the eventual data set? (I'm guessing very small, but jic...)

Yes, very small. A few times the number of labs instances, so -- a few thousand records unless I'm totally misunderstanding how this works.

Is the service visible/queryable to hosts outside WMF network, or entirely internal?

pdns will be a public dns server, although it will probably have some other recursors in front of it.

How does pdns relate to all the other databases supporting labs services on, eg, virt1000? Should we be choosing a physical production database server that will eventually handle pdns and [insert-other-stuff-here]?

Yes, as you can see with the dependency chain of this bug, ideally all nova services would use proper db services, and they may as well share this one.

I'm pretty sure that all your original guesses are compatible with this -- let me know if we need to back something out.

Thank you!

-Andrew

Change 197511 had a related patch set uploaded (by Springle):
Update m1 grants template to reflect reality.

https://gerrit.wikimedia.org/r/197511

I noticed two grants were applied on m1-master:

pdns_admin@dbproxy1001
pdns_admin@holmium

The holmium account only had INDEX priv. I've removed it.

Please only grant and connect via dbproxy1001 (m1-master.eqiad.wmnet CNAME points to dbproxy1001). The way @Joe did it was correct.

Change 197511 merged by Springle:
Update m1 grants template to reflect reality.

https://gerrit.wikimedia.org/r/197511

This is all working!