Page MenuHomePhabricator

Replace Exim on VRTS servers with Postfix
Open, MediumPublic

Description

Though the Exim servers on the VRTS hosts are not internet facing, it would be great to prioritize migrating them to Postfix. Per T232343 Postfix offers a better track record than Exim on security vulnerabilities.

  • Audit Current Setup
    • Incoming/outgoing mail handling rules
    • Queue management
  • Postfix Configuration Design
    • Port current exim config to postfix: transport and routing logic
    • Puppetization of the same
    • Rspam training in the same way as is currently done with Spamassassin (T402260)
  • Cutover Checklist
    • Draining current queues
    • Pause mail delivery to VRTS MX level?
    • Activate postfix and disable exim
  • Testing
  • Rollback Plan

Details

Other Assignee
Arnoldokoth
Related Changes in Gerrit:

Event Timeline

jhathaway triaged this task as Low priority.
LSobanski raised the priority of this task from Low to Medium.Jul 3 2025, 1:05 PM

would it be possible to setup a separate vrts server, that is configured with postfix, rather than replacing exim on the current server?

Hosts are not virtual, they are physical machines. So the biggest issue with that would be where to get hardware from (without a hardware request and the time/budget at that takes at least).

Maybe we could use the standby host to do it there and then at some point fail over between machines but that is also not completely without concerns because then we wouldn't have a real failover machine if we suddenly needed it. I guess it depends how long we would be in that state.

cc: @LSobanski

@Dzahn Or we could repurpose a spare server (if available)? miscweb comes to mind... Or were those VMs?

@Arnoldokoth If there is a spare server, sure, but I am not sure there is one. Back in the days dcops had a spare pool but at some point this changed and not sure now. We would have to ask them if they have any.

Generally a good idea, if something has been decom'ed lately but is still in the racks we might be able to use it for a bit.

But this would also means it's a temporary thing that still has to be migrated elsewhere after some time.

miscweb specifically were VMs.. so yea..not those..but maybe something different.

But another question comes to mind.. and that is.. do VRTS machines really need to be physical in general (anymore)?

Well... or we could create a VM and try to install VRTS with postfix on that. If that works (where I'm not sure how to actually test yet) it would be both the test for postfix and also a test if VRTS can just run on VMs. If it can.. we could flip over to that and get 2 physical servers back (maybe as our spares for other services).

But another question comes to mind.. and that is.. do VRTS machines really need to be physical in general (anymore)?

OTRS used to run on a VM, but IIRC was moved to hardware to address performance issues where more people started to use it. But for a test VM to explore the new mail setup, this should not matter.

@Dzahn We used to run it on VMs but we kept running into resource issues (especially with clamav) even after resizing the VM a few times. Additionally, it was also suggested that we could migrate the attachments to the disk though this is still on ice as we wait for object storage support in newer Znuny versions and compare (last I checked this was still in development).

+1 on using a VM to test but -1 on flipping over the physical hosts.

That said, if you striclty need a physical host for the tests, you could use puppetserver2003. I decommed it today in https://phabricator.wikimedia.org/T398607 and you could reclaim it. It has 128G RAM.

Thanks @MoritzMuehlenhoff We'll consider that... But I'm doubtful we "strictly" need to test this on hardware. A VM will just work as well me thinks. @jhathaway is out on vacation so we can wait and hear is thoughts when he's back.

Thanks all. I am not sure though if the request was for "temp testing setup" or just for "a new system to replace the old".

Thanks @MoritzMuehlenhoff We'll consider that... But I'm doubtful we "strictly" need to test this on hardware. A VM will just work as well me thinks. @jhathaway is out on vacation so we can wait and hear is thoughts when he's back.

Testing on a vm should work fine

Change #1178874 had a related patch set uploaded (by AOkoth; author: AOkoth):

[operations/puppet@production] vrts: Create test role and profile

https://gerrit.wikimedia.org/r/1178874