Page MenuHomePhabricator

Allow interacting with Toolforge PuppetDB from wmcs-cookbooks
Open, LowPublic4 Estimated Story Points

Description

While most of Cloud VPS does not use PuppetDB, the Toolforge and toolsbeta projects do have a PuppetDB server hooked up to the project-local Puppet servers. If we could query those in cumin/cookbooks then implementing features like T356970: toolforge k8s automation: introduce option to reboot a node if the uptime is higher than XYZ would be more efficient.

I think a major issue here atm is that the Cumin PuppetDB backend can only interact with a single PuppetDB server, while for this we would need at least two (one for tools and one for toolsbeta), plus any future WMCS managed projects that have a local PuppetDB server.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

The change would not be very small as to make it general we would need to make cumin support multiple instances of each backend, each one with their own settings and also a way to select them via the query language. Definitely a breaking change for the existing config and query language.

As a temporary and also a bit unpleasant workaround you could have puppet generate multiple cumin configuration files and use a different file based on the cluster you want to work on. Ditto for spicerack setting the instance_params's cumin_config to the different ones. That would require though to run cookbooks with the -c/--config-file flag.

Volans moved this task from Backlog to Nice to have on the Cumin board.
Volans set the point value for this task to 4.

I just discovered that we do have dedicated Cumin instances for tools and toolsbeta, where /etc/cumin/config.yaml points to the respective puppetdb instead of the production one:

fnegri@tools-cumin-1:~$ sudo cat /etc/cumin/config.yaml
puppetdb:
    host: tools-puppetdb-2.tools.eqiad1.wikimedia.cloud

So you can make queries such as:

fnegri@tools-cumin-1:~$ sudo cumin 'P{O:wmcs::db::toolsdb}'
2 hosts will be targeted:
tools-db-[1,3].tools.eqiad1.wikimedia.cloud