Page MenuHomePhabricator
Create Task
Maniphest T212595

[Feature request] Database as a Service (Trove) for Cloud VPS projects
Closed, ResolvedPublic
Actions

Description

Trove is a nice service that can provide database services to the cloud. So I, a tool developer, can build my own database, use it in services and easily throw away my VMs when needed without fear of losing data. It would also improves security of the databases.

Details

ProjectBranchLines +/-Subject
operations/puppetproduction+16 -0Trove: add support config for mariadb
operations/puppetproduction+9 -0Trove config: add some configs for redis and postgres
openstack/horizon/deploymain+1 -2Install/enable Trove dashboard
operations/puppetproduction+38 -11OpenStack Trove: enable dns integration for DB access
operations/puppetproduction+12 -15Trove: allow labs instances to talk to rabbitmq
operations/puppetproduction+9 -0Trove: set up haproxy for trove-api in eqiad1
operations/puppetproduction+53 -0Trove: puppetize api-paste.ini
operations/puppetproduction+31 -0Deploy OpenStack Trove in eqiad1
openstack/horizon/trove-dashboardmain+0 -97Remove 'backup' panels
operations/puppetproduction+2 -0Trove: specify the mysql container should come from quay.io
operations/puppetproduction+7 -0Trove: use our quay.io docker registry rather than docker hub.
operations/puppetproduction+6 -4Trove: set low default quotas per project but big potential DB size
operations/puppetproduction+24 -4Trove: move most trove activity into a service project
labs/privatemaster+2 -0Dummy passwords for Trove service user
operations/puppetproduction+15 -0Trove.conf: add [network] section
operations/puppetproduction+10 -16Update trove.conf as per Ussuri release notes
operations/puppetproduction+1 -1Install trove services in codfw1dev
openstack/horizon/deploymain+13 -2Added trove-dashboard submodule
openstack/horizon/wheelsmain+0 -0Add wheels needed by trove dashboard
operations/puppetproduction+1 -1codfw1dev: turn off trove for now
operations/puppetproduction+2 K -0Openstack Trove: Hack in a bugfix that's missing from the debian package
operations/puppetproduction+6 -6OpenStack Trove: use /dev/sdb instead of /dev/vdb
operations/puppetproduction+1 -1trove-guestagent.conf: Don't set log_config_append
operations/puppetproduction+15 -0Rabbitmq: open firewall to traffic from the cloud in codfw1dev
operations/puppetproduction+1 -0cloud-vps: include python3-troveclient
operations/puppetproduction+703 -0Rough in OpenStack Trove module
labs/privatemaster+4 -0Add fake passwords for OpenStack Trove
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Ladsgroup mentioned this in T255046: Request creation of "chat" VPS project (for a Mattermost instance).Jun 21 2020, 12:49 AM
taavi added a subscriber: taavi.Jun 27 2020, 10:13 AM
Bstorm added a parent task: T136335: Allow self-serve database credential and permissions management for Toolforge projects.Oct 9 2020, 3:59 PM
Bstorm added a subtask: T236101: Find a way to remove non-replicated tables from ToolsDB.Oct 9 2020, 4:08 PM
Bstorm added a subtask: T188406: Provide access to user created databases in PAWS.
Bstorm removed a subtask: T188406: Provide access to user created databases in PAWS.Oct 9 2020, 4:10 PM
Bstorm added a parent task: T188406: Provide access to user created databases in PAWS.
Bstorm removed a subtask: T236101: Find a way to remove non-replicated tables from ToolsDB.
Bstorm added a parent task: T236101: Find a way to remove non-replicated tables from ToolsDB.
Bstorm mentioned this in T267683: Move PAWS to it's own database (and away from ToolsDB).Nov 10 2020, 8:53 PM
Bstorm raised the priority of this task from Low to Medium.Nov 10 2020, 9:51 PM
Bstorm mentioned this in T266587: ToolsDB replication is broken.Nov 11 2020, 7:47 PM
Andrew added a subscriber: Andrew.Mar 9 2021, 4:37 PM

FYI @Ladsgroup, I have investigating this feature as one of my quarterly goals. I can't yet predict when it will happen but it's definitely something we all want.

Andrew renamed this task from [Feature request] Database as a Service for Cloud VPS projects to [Feature request] Database as a Service (Trove) for Cloud VPS projects.Mar 9 2021, 4:37 PM
Ladsgroup added a comment.Mar 9 2021, 7:48 PM

Thank you. Looking forward to it!

aborrero closed subtask T278505: Prepare/import debian packages for openstack trove as Resolved.Mar 30 2021, 2:33 PM
Andrew claimed this task.Mar 30 2021, 4:02 PM
gerritbot added a comment.Mar 30 2021, 4:54 PM

Change 675842 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] Rough in OpenStack Trove module

https://gerrit.wikimedia.org/r/675842

gerritbot added a project: Patch-For-Review.Mar 30 2021, 4:54 PM
gerritbot added a comment.Mar 30 2021, 5:57 PM

Change 675851 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[labs/private@master] Add fake passwords for OpenStack Trove

https://gerrit.wikimedia.org/r/675851

gerritbot added a comment.Mar 30 2021, 5:58 PM

Change 675851 merged by Andrew Bogott:
[labs/private@master] Add fake passwords for OpenStack Trove

https://gerrit.wikimedia.org/r/675851

Andrew mentioned this in rLPRI670c78b5f045: Add fake passwords for OpenStack Trove.Mar 30 2021, 6:01 PM
gerritbot added a comment.Mar 30 2021, 6:20 PM

Change 675842 merged by Andrew Bogott:
[operations/puppet@production] Rough in OpenStack Trove module

https://gerrit.wikimedia.org/r/675842

gerritbot added a comment.Mar 30 2021, 6:32 PM

Change 675858 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):
[operations/puppet@production] cloud-vps: include python3-troveclient

https://gerrit.wikimedia.org/r/675858

gerritbot added a comment.Mar 30 2021, 6:32 PM

Change 675858 merged by Andrew Bogott:
[operations/puppet@production] cloud-vps: include python3-troveclient

https://gerrit.wikimedia.org/r/675858

Maintenance_bot removed a project: Patch-For-Review.Mar 30 2021, 7:12 PM
gerritbot added a comment.Mar 30 2021, 8:25 PM

Change 675899 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Rabbitmq: open firewall to traffic from the cloud in codfw1dev

https://gerrit.wikimedia.org/r/675899

gerritbot added a project: Patch-For-Review.Mar 30 2021, 8:25 PM

Change 675899 merged by Andrew Bogott:

[operations/puppet@production] Rabbitmq: open firewall to traffic from the cloud in codfw1dev

https://gerrit.wikimedia.org/r/675899

gerritbot added a comment.Mar 30 2021, 9:10 PM

Change 675909 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] trove-guestagent.conf: Don't set log_config_append

https://gerrit.wikimedia.org/r/675909

gerritbot added a comment.Mar 30 2021, 9:12 PM

Change 675909 merged by Andrew Bogott:

[operations/puppet@production] trove-guestagent.conf: Don't set log_config_append

https://gerrit.wikimedia.org/r/675909

gerritbot added a comment.Mar 31 2021, 7:16 PM

Change 676135 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] OpenStack Trove: use /dev/sdb instead of /dev/vdb

https://gerrit.wikimedia.org/r/676135

gerritbot added a comment.Mar 31 2021, 7:16 PM

Change 676137 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Openstack Trove: Hack in a bugfix that's missing from the debian package

https://gerrit.wikimedia.org/r/676137

gerritbot added a comment.Mar 31 2021, 7:17 PM

Change 676135 merged by Andrew Bogott:

[operations/puppet@production] OpenStack Trove: use /dev/sdb instead of /dev/vdb

https://gerrit.wikimedia.org/r/676135

gerritbot added a comment.Mar 31 2021, 7:22 PM

Change 676137 merged by Andrew Bogott:

[operations/puppet@production] Openstack Trove: Hack in a bugfix that's missing from the debian package

https://gerrit.wikimedia.org/r/676137

Andrew added a comment.Mar 31 2021, 11:07 PM

General agreement is that we should wait until version Victoria to actually run this as a production service. In the meantime I'll keep the puppet code around and forward it on through U to V.

gerritbot added a comment.Mar 31 2021, 11:08 PM

Change 676169 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] codfw1dev: turn off trove for now

https://gerrit.wikimedia.org/r/676169

gerritbot added a comment.Mar 31 2021, 11:49 PM

Change 676169 merged by Andrew Bogott:

[operations/puppet@production] codfw1dev: turn off trove for now

https://gerrit.wikimedia.org/r/676169

gerritbot added a comment.Apr 15 2021, 6:40 PM

Change 679918 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[openstack/horizon/wheels@main] Add wheels needed by trove dashboard

https://gerrit.wikimedia.org/r/679918

gerritbot added a comment.Apr 15 2021, 6:40 PM

Change 679919 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[openstack/horizon/deploy@main] Added trove-dashboard submodule

https://gerrit.wikimedia.org/r/679919

gerritbot added a comment.Apr 15 2021, 6:41 PM

Change 679918 merged by Andrew Bogott:

[openstack/horizon/wheels@main] Add wheels needed by trove dashboard

https://gerrit.wikimedia.org/r/679918

gerritbot added a comment.Apr 15 2021, 6:45 PM

Change 679919 merged by Andrew Bogott:

[openstack/horizon/deploy@main] Added trove-dashboard submodule

https://gerrit.wikimedia.org/r/679919

gerritbot added a comment.Apr 15 2021, 7:08 PM

Change 679924 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Install trove services in codfw1dev

https://gerrit.wikimedia.org/r/679924

gerritbot added a comment.Apr 15 2021, 7:09 PM

Change 679924 merged by Andrew Bogott:

[operations/puppet@production] Install trove services in codfw1dev

https://gerrit.wikimedia.org/r/679924

gerritbot added a comment.Apr 16 2021, 2:53 AM

Change 680102 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Update trove.conf as per Ussuri release notes

https://gerrit.wikimedia.org/r/680102

gerritbot added a comment.Apr 16 2021, 2:54 AM

Change 680102 merged by Andrew Bogott:

[operations/puppet@production] Update trove.conf as per Ussuri release notes

https://gerrit.wikimedia.org/r/680102

aborrero closed subtask T280310: communication on port 5672 (amqp) between guest instances and cloudcontrols as Resolved.Apr 20 2021, 4:34 PM
gerritbot added a comment.Apr 21 2021, 10:04 PM

Change 681790 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Trove.conf: add [network] section

https://gerrit.wikimedia.org/r/681790

gerritbot added a comment.Apr 21 2021, 10:05 PM

Change 681790 merged by Andrew Bogott:

[operations/puppet@production] Trove.conf: add [network] section

https://gerrit.wikimedia.org/r/681790

gerritbot added a comment.Apr 23 2021, 1:28 PM

Change 682126 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[labs/private@master] Dummy passwords for Trove service user

https://gerrit.wikimedia.org/r/682126

gerritbot added a comment.Apr 23 2021, 1:37 PM

Change 682129 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Trove: move most trove activity into a service project

https://gerrit.wikimedia.org/r/682129

gerritbot added a comment.Apr 23 2021, 1:37 PM

Change 682126 merged by Andrew Bogott:

[labs/private@master] Dummy passwords for Trove service user

https://gerrit.wikimedia.org/r/682126

Andrew mentioned this in rLPRI6c7119236e74: Dummy passwords for Trove service user.Apr 23 2021, 1:38 PM
gerritbot added a comment.Apr 23 2021, 2:04 PM

Change 682129 merged by Andrew Bogott:

[operations/puppet@production] Trove: move most trove activity into a service project

https://gerrit.wikimedia.org/r/682129

Andrew closed subtask T280835: Wrong version of Trove (13) in Victoria osbpo as Resolved.Apr 23 2021, 5:46 PM
Andrew added a subscriber: Bstorm.

Trove is now up and running in codfw1dev. Going to pause development for now until I get some user feedback from @Bstorm

gerritbot added a comment.Apr 27 2021, 9:48 PM

Change 683092 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Trove: set low default quotas per project.

https://gerrit.wikimedia.org/r/683092

Andrew moved this task from Inbox to Needs discussion on the cloud-services-team (Kanban) board.Apr 28 2021, 3:34 PM
Andrew added a comment.EditedApr 28 2021, 3:56 PM

Next topic: Where does Trove get its docker images? We should get them from quay.io or self-host.

gerritbot added a comment.Apr 29 2021, 2:08 PM

Change 683092 merged by Andrew Bogott:

[operations/puppet@production] Trove: set low default quotas per project but big potential DB size

https://gerrit.wikimedia.org/r/683092

gerritbot added a comment.Apr 30 2021, 4:15 PM

Change 683924 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Trove: use our quay.io docker registry rather than docker hub.

https://gerrit.wikimedia.org/r/683924

gerritbot added a comment.Apr 30 2021, 4:36 PM

Change 683924 merged by Andrew Bogott:

[operations/puppet@production] Trove: use our quay.io docker registry rather than docker hub.

https://gerrit.wikimedia.org/r/683924

gerritbot added a comment.Apr 30 2021, 4:52 PM

Change 683928 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Trove: specify the mysql container should come from quay.io

https://gerrit.wikimedia.org/r/683928

gerritbot added a comment.Apr 30 2021, 4:54 PM

Change 683928 merged by Andrew Bogott:

[operations/puppet@production] Trove: specify the mysql container should come from quay.io

https://gerrit.wikimedia.org/r/683928

Andrew added a comment.May 2 2021, 10:10 PM

From irc:

andrewbogott: you can use images there at your own responsibility, by default, root acount is disabled and trove admin should config nova keypair for trove instances.

gerritbot added a comment.May 7 2021, 2:31 PM

Change 686611 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[openstack/horizon/trove-dashboard@main] Remove 'backup' panels

https://gerrit.wikimedia.org/r/686611

gerritbot added a comment.May 7 2021, 2:32 PM

Change 686611 merged by Andrew Bogott:

[openstack/horizon/trove-dashboard@main] Remove 'backup' panels

https://gerrit.wikimedia.org/r/686611

Andrew closed subtask T276956: Make a plan for Trove v. rabbit as Resolved.May 10 2021, 8:51 PM
Andrew closed subtask T276960: Questions about Trove guest images as Resolved.
aborrero closed subtask T281654: Trove-dashboard: web app crashes when a db instance is deleted. as Resolved.May 11 2021, 4:11 PM
gerritbot added a comment.May 11 2021, 5:48 PM

Change 689190 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Deploy OpenStack Trove in eqiad1

https://gerrit.wikimedia.org/r/689190

gerritbot added a comment.May 11 2021, 6:31 PM

Change 689190 merged by Andrew Bogott:

[operations/puppet@production] Deploy OpenStack Trove in eqiad1

https://gerrit.wikimedia.org/r/689190

gerritbot added a comment.May 11 2021, 6:50 PM

Change 689200 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Trove: puppetize api-paste.ini

https://gerrit.wikimedia.org/r/689200

gerritbot added a comment.May 11 2021, 6:51 PM

Change 689200 merged by Andrew Bogott:

[operations/puppet@production] Trove: puppetize api-paste.ini

https://gerrit.wikimedia.org/r/689200

gerritbot added a comment.May 11 2021, 6:55 PM

Change 689201 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Trove: set up haproxy for trove-api in eqiad1

https://gerrit.wikimedia.org/r/689201

gerritbot added a comment.May 11 2021, 7:00 PM

Change 689201 merged by Andrew Bogott:

[operations/puppet@production] Trove: set up haproxy for trove-api in eqiad1

https://gerrit.wikimedia.org/r/689201

gerritbot added a comment.May 11 2021, 8:23 PM

Change 689266 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Trove: allow labs instances to talk to rabbitmq

https://gerrit.wikimedia.org/r/689266

gerritbot added a comment.May 11 2021, 8:32 PM

Change 689266 merged by Andrew Bogott:

[operations/puppet@production] Trove: allow labs instances to talk to rabbitmq

https://gerrit.wikimedia.org/r/689266

Andrew moved this task from Needs discussion to Doing on the cloud-services-team (Kanban) board.May 12 2021, 2:54 PM
gerritbot added a comment.May 13 2021, 3:24 PM

Change 690546 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] OpenStack Trove: enable dns integration for DB access

https://gerrit.wikimedia.org/r/690546

gerritbot added a comment.May 13 2021, 3:58 PM

Change 690546 merged by Andrew Bogott:

[operations/puppet@production] OpenStack Trove: enable dns integration for DB access

https://gerrit.wikimedia.org/r/690546

gerritbot added a comment.May 13 2021, 5:14 PM

Change 690581 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[openstack/horizon/deploy@main] Install/enable Trove dashboard

https://gerrit.wikimedia.org/r/690581

gerritbot added a comment.May 13 2021, 5:15 PM

Change 690581 merged by Andrew Bogott:

[openstack/horizon/deploy@main] Install/enable Trove dashboard

https://gerrit.wikimedia.org/r/690581

Andrew closed subtask T282798: Trove: don't permit hard affinity within DB clusters as Resolved.May 14 2021, 2:10 PM
taavi added a parent task: T267683: Move PAWS to it's own database (and away from ToolsDB).May 15 2021, 11:20 AM
Andrew closed subtask T282809: Allow access to Trove API endpoints (port 8779) from cloud-vps instances as Resolved.May 20 2021, 2:34 PM
gerritbot added a comment.Jun 3 2021, 7:00 PM

Change 698017 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Trove config: add some configs for redis and postgres

https://gerrit.wikimedia.org/r/698017

gerritbot added a comment.Jun 3 2021, 7:04 PM

Change 698017 merged by Andrew Bogott:

[operations/puppet@production] Trove config: add some configs for redis and postgres

https://gerrit.wikimedia.org/r/698017

gerritbot added a comment.Jun 3 2021, 8:20 PM

Change 698051 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Trove: add support config for mariadb

https://gerrit.wikimedia.org/r/698051

gerritbot added a comment.Jun 3 2021, 8:21 PM

Change 698051 merged by Andrew Bogott:

[operations/puppet@production] Trove: add support config for mariadb

https://gerrit.wikimedia.org/r/698051

taavi added a subtask: T286720: trove: Unable to manage databases on mariadb 10.6.1.Jul 15 2021, 1:56 PM
Andrew closed subtask T286720: trove: Unable to manage databases on mariadb 10.6.1 as Resolved.Jul 16 2021, 4:48 PM
Andrew closed subtask T283299: Add postgresql datastore to Trove as Resolved.Jul 19 2021, 8:56 PM
Andrew closed this task as Resolved.Aug 24 2021, 9:30 PM

This is released!

Bstorm added a subtask: T290349: Trove configurations aren't working.Sep 3 2021, 6:25 PM
Bstorm closed subtask T290349: Trove configurations aren't working as Resolved.Sep 3 2021, 7:22 PM
taavi removed a parent task: T236101: Find a way to remove non-replicated tables from ToolsDB.Nov 7 2021, 5:51 PM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL