This is a task for failing over otrs1001 (buster) to vrts1001(bullseye) in eqiad.
- Create scheduled maintenance window on VRTS dashboard to notify users of upcoming downtime
- Downtime otrs1001 for two hours on cumin:
sudo cookbook sre.hosts.downtime -r "Replacing Host" -H 2 otrs1001.eqiad.wmnet
- Disable Puppet on otrs1001
- Disable all services on otrs1001:
sudo systemctl stop cron sudo systemctl stop exim4 sudo systemctl stop apache2 sudo systemctl stop vrts-daemon
- Create patch and merge with the following changes:
- Applying vrts role on vrts1001
- Changing active_host on Puppet hiera to vrts1001.eqiad.wmnet.
- Run sudo /usr/local/bin/install_vrts 6.0.48 on vrts1001
- Run sudo -u www-data /opt/otrs/bin/otrs.Console.pl Admin::Package::ReinstallAll
- Ensure all services are running normally on vrts1001:
sudo systemctl status cron sudo systemctl status exim4 sudo systemctl status apache2 sudo systemctl status vrts-daemon
- Create patch in DNS repo. In the wmnet template, point ticket to vrts1001.eqiad.wmnet. Merge and run authdns-update
- Access https://ticket.wikimedia.org and login and check if everything is working normally.
If everything is working fine, extend downtime on otrs1001 and prepare for decom.