While auditing codfw/eqiad traffic during switchover (T286038) I came across plaintext rsync for deploy hosts, please consider switching to encrypted rsync
Description
Details
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | fgiunchedi | T286038 Record traffic flows in and out of eqiad during switchover | |||
| Resolved | Clement_Goubert | T289857 Use encrypted rsync for deployment::rsync |
Event Timeline
@fgiunchedi do you have any pointers on what switching to encrypted rsync entails? Is it just a puppet setting somewhere?
Essentially a puppet setting yes, rsync::server::wrap_with_stunnel for the server bits and then e.g. rsync::quickdatacopy has the option to turn on ssl on the client too. See e.g. grafana and netmon use encrypted rsync
Change 715638 had a related patch set uploaded (by Legoktm; author: Legoktm):
[operations/puppet@production] [WIP] deployment: Use rsync::quickdatacopy, enable encryption
I'm guessing no one has done this until now because deployment::rsync was using hand-rolled rsync + timer rather than quickdatacopy. I gave it a shot at migrating to that, which seemed better long-term than copying all the stunnel encryption stuff.
I don't think this should be considered a blocker for T327920: March 2023 Datacenter Switchover.
Mentioned in SAL (#wikimedia-operations) [2023-06-28T10:51:31Z] <claime> Migrating to rsync::quickdatacopy for deployment servers - T289857
Change 715638 merged by Clément Goubert:
[operations/puppet@production] deployment: Use rsync::quickdatacopy, enable encryption
Mentioned in SAL (#wikimedia-operations) [2023-06-28T11:08:00Z] <claime> Reverting migration to rsync::quickdatacopy for deployment servers - T289857
Reverted because rsync::quickdatacopy wants fqdns, we're giving it IPs, nothing gets deployed.
I will prepare a fix and we can try again.
Change 933911 had a related patch set uploaded (by Clément Goubert; author: Legoktm):
[operations/puppet@production] deployment: Use rsync::quickdatacopy, enable encryption
Change 933911 merged by Clément Goubert:
[operations/puppet@production] deployment: Use rsync::quickdatacopy, enable encryption
Mentioned in SAL (#wikimedia-operations) [2023-07-04T14:53:55Z] <claime> Deploying encrypted rsync to deployment servers - T289857