The upcoming version of nova uses a new abstraction layer for instance tracking ('cells v2') which requires an additional database. It should be called 'nova_cell0' and have the same access grants as the existing 'nova' database.
I can definitely set this up myself, but perhaps the DBAs would prefer that I not do that :)
Update: the database is finally called nova_cell0_eqiad1
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Andrew | T237749 Upgrade wmcs OpenStack version to Ocata | |||
| Resolved | Andrew | T239160 nova: set up cell and host mappings | |||
| Resolved | Marostegui | T239170 Create a new nova database on m5 named 'nova_cell0' |
So I would assume the steps needed are:
Those steps sound right to me. Backups would be nice -- I think the nova db is backed up but I'm not positive.
Does this need backups? cc @jcrespo
To add misc hosts to backups (assuming they are existing backed hosts), the only thing needed is to provide SELECT GRANTS on all objects of the database to the backup user.
Yep, thanks - I can add those too, unless you prefer to do it yourself. Either way is fine with me. It was more a question for @Andrew (which he already answered) and CCing you so you could be aware.
Thank you both
It was more a question for @Andrew (which he already answered)
I commented also so he also is aware and he could even do it himself if he doesn't want to get blocked on us. 0:-)
The process wasn't well documented, so I added it explicitly at https://wikitech.wikimedia.org/wiki/MariaDB/Backups#Adding_a_new_dump so now anyone can do it.
@Andrew I was checking the grants for the nova database and there are no grants defined on production-m5.sql.erb for that given database.
However, the database does have:
GRANT ALL PRIVILEGES ON `nova`.* TO 'nova'@'%'
That is not reflected on the file, so we should add it.
Also, can we maybe be more specific about that grant and use certain IPs instead of using %? For either nova and the new nova_cell0
Also, can we maybe be more specific about that grant and use certain IPs instead of using %? For either nova and the new nova_cell0
The only hosts that talk directly to the nova database (and the nova-api database) are cloudcontrol1003 (208.80.154.23) and cloudcontrol1004 (208.80.154.132)
Change 553350 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] production-m5.sql: Add access to nova_cell0 for nova user
Mentioned in SAL (#wikimedia-operations) [2019-11-27T14:50:41Z] <marostegui> Create nova_cell0 database on m5 master - T239170
Database created:
root@db1133.eqiad.wmnet[(none)]> create database if not exists nova_cell0; Query OK, 1 row affected (0.01 sec) root@db1133.eqiad.wmnet[(none)]> show databases like 'nova%' ; +------------------+ | Database (nova%) | +------------------+ | nova | | nova_api | | nova_api_eqiad1 | | nova_cell0 | | nova_eqiad1 | +------------------+ 5 rows in set (0.00 sec)
I have created the grants for cloudcontrol hosts for nova user, so we can remove 'nova'@'%' too:
root@db1133.eqiad.wmnet[(none)]> show grants for 'nova'@'208.80.154.23'; show grants for 'nova'@'208.80.154.132'; +-----------------------------------------------------------------------------------------------------------------------------------------------+ | Grants for nova@208.80.154.23 | +-----------------------------------------------------------------------------------------------------------------------------------------------+ | GRANT USAGE ON *.* TO 'nova'@'208.80.154.23' IDENTIFIED BY PASSWORD 'xx' WITH MAX_USER_CONNECTIONS 100 | | GRANT ALL PRIVILEGES ON `nova`.* TO 'nova'@'208.80.154.23' | | GRANT ALL PRIVILEGES ON `nova_eqiad1`.* TO 'nova'@'208.80.154.23' | | GRANT ALL PRIVILEGES ON `nova_api_eqiad1`.* TO 'nova'@'208.80.154.23' | +-----------------------------------------------------------------------------------------------------------------------------------------------+ 4 rows in set (0.00 sec) +------------------------------------------------------------------------------------------------------------------------------------------------+ | Grants for nova@208.80.154.132 | +------------------------------------------------------------------------------------------------------------------------------------------------+ | GRANT USAGE ON *.* TO 'nova'@'208.80.154.132' IDENTIFIED BY PASSWORD 'xx' WITH MAX_USER_CONNECTIONS 100 | | GRANT ALL PRIVILEGES ON `nova`.* TO 'nova'@'208.80.154.132' | | GRANT ALL PRIVILEGES ON `nova_api_eqiad1`.* TO 'nova'@'208.80.154.132' | | GRANT ALL PRIVILEGES ON `nova_eqiad1`.* TO 'nova'@'208.80.154.132' | +------------------------------------------------------------------------------------------------------------------------------------------------+ 4 rows in set (0.00 sec)
Change 553350 merged by Marostegui:
[operations/puppet@production] production-m5.sql: Add access to nova_cell0 for nova user
Mentioned in SAL (#wikimedia-operations) [2019-11-27T14:56:20Z] <marostegui> Add new grants for nova_cell0 database on m5 - T239170
Grants added for nova_cell0:
root@db1133.eqiad.wmnet[(none)]> show grants for 'nova'@'208.80.154.23'; show grants for 'nova'@'208.80.154.132'; | GRANT ALL PRIVILEGES ON `nova_cell0`.* TO 'nova'@'208.80.154.23' | 5 rows in set (0.00 sec) | GRANT ALL PRIVILEGES ON `nova_cell0`.* TO 'nova'@'208.80.154.132' | 5 rows in set (0.00 sec)
@Andrew @JHedden can you confirm if it is accessible from the cloudcontrol hosts?
Change 553351 had a related patch set uploaded (by Marostegui; owner: Marostegui):
[operations/puppet@production] production-m5.sql.erb: Change nova_cell0 to nova_cell0_eqiad1
Change 553351 merged by Marostegui:
[operations/puppet@production] production-m5.sql.erb: Change nova_cell0 to nova_cell0_eqiad1
@Andrew requested a different name for the database, so this is the new one with the desired charset/collation:
root@db1133.eqiad.wmnet[nova_eqiad1]> create database if not exists nova_cell0_eqiad1 character set utf8 collate utf8_general_ci ; Query OK, 1 row affected (0.00 sec) root@db1133.eqiad.wmnet[nova_eqiad1]> show create database nova_cell0_eqiad1; +-------------------+----------------------------------------------------------------------------+ | Database | Create Database | +-------------------+----------------------------------------------------------------------------+ | nova_cell0_eqiad1 | CREATE DATABASE `nova_cell0_eqiad1` /*!40100 DEFAULT CHARACTER SET utf8 */ | +-------------------+----------------------------------------------------------------------------+ 1 row in set (0.00 sec)
Grants added too:
root@db1133.eqiad.wmnet[nova_eqiad1]> pager grep nova_cell0; PAGER set to 'grep nova_cell0' root@db1133.eqiad.wmnet[nova_eqiad1]> show grants for 'nova'@'208.80.154.23'; show grants for 'nova'@'208.80.154.132'; | GRANT ALL PRIVILEGES ON `nova_cell0_eqiad1`.* TO 'nova'@'208.80.154.23' | 5 rows in set (0.00 sec) | GRANT ALL PRIVILEGES ON `nova_cell0_eqiad1`.* TO 'nova'@'208.80.154.132' | 5 rows in set (0.00 sec)
@Andrew can you confirm you can access the new database?
Mentioned in SAL (#wikimedia-operations) [2019-11-27T15:27:41Z] <marostegui> Add grants for dump (10.64.0.95,10.64.16.31) for nova_cell0_eqiad database on db1117:3325 and db2078:3325 - T239170
Mentioned in SAL (#wikimedia-operations) [2019-11-27T15:29:10Z] <marostegui> Add grants for dump (10.192.0.114,10.192.16.96) for nova_cell0_eqiad database on db1117:3325 and db2078:3325 - T239170
Mentioned in SAL (#wikimedia-operations) [2019-11-27T15:37:07Z] <marostegui> Logging retroactively for the record: drop user 'nova'@'%' from m5 - T239170
Grants for nova_cell0_eqiad1 added to the dump users
Closing this task, please re-open if needed
Thanks everyone!