Page MenuHomePhabricator

Install cumin in the WMCS infrastructure
Closed, ResolvedPublic

Description

As part of the parent task effort to deprecate and remove Salt from out infrastructure, we need an alternative way for the WMCS admins to reach all the instances and Cumin can be used for that.

This task is to track the installation of Cumin into the WMCS infrastructure and ensure it works and that Cumin masters can reach and connect to all instances.

As of now all the tests will be performed with the direct backend using FQDN, while the development of the dedicated backend for OpenStack is done as part of T175711

Event Timeline

Volans created this task.Sep 12 2017, 4:43 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 12 2017, 4:43 PM
Volans moved this task from Backlog to In Progress on the SRE-tools board.Sep 12 2017, 4:44 PM
hashar added a subscriber: hashar.Sep 12 2017, 6:48 PM

As a side effect, Beta-Cluster-Infrastructure and Continuous-Integration-Infrastructure would need a way to have a per project cumin master. We don't have access to the WMCS salt master.

The instances are:

deployment-salt02.deployment-prep.eqiad.wmflabs
integration-saltmaster.integration.eqiad.wmflabs

Change 377787 had a related patch set uploaded (by Volans; owner: Volans):
[operations/puppet@production] WMCS: install Cumin for WMCS admins

https://gerrit.wikimedia.org/r/377787

Change 377796 had a related patch set uploaded (by Volans; owner: Volans):
[labs/private@master] keyholder: add Cumin openstack master key

https://gerrit.wikimedia.org/r/377796

Change 377796 merged by Volans:
[labs/private@master] keyholder: add Cumin openstack master key

https://gerrit.wikimedia.org/r/377796

Volans moved this task from In Progress to In Code Review on the SRE-tools board.Sep 13 2017, 4:31 PM

As a side effect, Beta-Cluster-Infrastructure and Continuous-Integration-Infrastructure would need a way to have a per project cumin master. We don't have access to the WMCS salt master.

The instances are:

deployment-salt02.deployment-prep.eqiad.wmflabs
integration-saltmaster.integration.eqiad.wmflabs

what does integration (CI) use salt for?

Change 377787 merged by Andrew Bogott:
[operations/puppet@production] WMCS: install Cumin for WMCS admins

https://gerrit.wikimedia.org/r/377787

Mentioned in SAL (#wikimedia-operations) [2017-09-19T21:51:31Z] <volans> disabled puppet on labpuppetmaster1001, testing cumin T175712

Change 379192 had a related patch set uploaded (by Volans; owner: Volans):
[operations/puppet@production] WMCS: fix Cumin configuration

https://gerrit.wikimedia.org/r/379192

As a side effect, Beta-Cluster-Infrastructure and Continuous-Integration-Infrastructure would need a way to have a per project cumin master. We don't have access to the WMCS salt master.

The instances are:

deployment-salt02.deployment-prep.eqiad.wmflabs
integration-saltmaster.integration.eqiad.wmflabs

what does integration (CI) use salt for?

For both beta and CI, we use it for mass commands execution. Be it checking state of instances, manually upgrading packages, mass clean up or deploying some scripts.

I guess that can be replaced by either:

  • cumin master (if there is a puppet class for standalone cumin master in labs)
  • cluster shell (apparently used by tools labs)

Change 379192 merged by Volans:
[operations/puppet@production] WMCS: fix Cumin configuration

https://gerrit.wikimedia.org/r/379192

Change 379209 had a related patch set uploaded (by Volans; owner: Volans):
[operations/puppet@production] WMCS: fine tuning Cumin configuration for proxies

https://gerrit.wikimedia.org/r/379209

Change 379209 merged by Volans:
[operations/puppet@production] WMCS: fine tuning Cumin configuration for proxies

https://gerrit.wikimedia.org/r/379209

As a side effect, Beta-Cluster-Infrastructure and Continuous-Integration-Infrastructure would need a way to have a per project cumin master. We don't have access to the WMCS salt master.

The instances are:

deployment-salt02.deployment-prep.eqiad.wmflabs
integration-saltmaster.integration.eqiad.wmflabs

@hashar I think this is unrelated with this task. My understanding is that those two WMCS projects have their own salt master internal to the project that they self-administer. Nothing forbid to keep it that way, and replacing the global WMCS salt master with cumin doesn't affect those in any way AFAIK.
I agree that it would be nice to have a simple way to install a cumin master inside a labs project, I can look into it once the goal-related work will be completed.
Feel free to open a dedicated task for it.

Volans closed this task as Resolved.Sep 20 2017, 11:48 AM
Volans removed a project: Patch-For-Review.

Cumin master is now installed on labpuppetmaster100[1-2].wikimedia.org and is able to connect via SSH to cloud instances, including the SSH proxy themselves. The targeting for the moment is done just using FQDNs with the direct backend, until T175711 is completed, in the next couple of days.

But as far as concern this specific task, the work is completed, so resolving it.

As a side effect, Beta-Cluster-Infrastructure and Continuous-Integration-Infrastructure would need a way to have a per project cumin master. We don't have access to the WMCS salt master.

The instances are:

deployment-salt02.deployment-prep.eqiad.wmflabs
integration-saltmaster.integration.eqiad.wmflabs

@hashar I think this is unrelated with this task. My understanding is that those two WMCS projects have their own salt master internal to the project that they self-administer. Nothing forbid to keep it that way, and replacing the global WMCS salt master with cumin doesn't affect those in any way AFAIK.
I agree that it would be nice to have a simple way to install a cumin master inside a labs project, I can look into it once the goal-related work will be completed.
Feel free to open a dedicated task for it.

Thanks. I have filled T176314: Replace salt on integration and deployment-prep projects