Page MenuHomePhabricator

reinstall RT server with private IP and Buster
Closed, ResolvedPublic

Description

RT (request tracker) is behind misc-web but on ununpentium.wikimedia.org

Nowdays it doesn't need a public IP anymore since it does not handle email anymore. We still need it but just as a readonly archive.

Create a new VM with a private IP (eqiad) (and stretch while touching it). Migrate RT to that. Fix issues with stretch. Adjust the misc-web director to the private IP.

Once done remove the old jessie VM and kill it from .wikimedia.org DNS zone.

Event Timeline

Dzahn triaged this task as Low priority.
Dzahn raised the priority of this task from Low to Medium.Sep 21 2018, 12:48 AM

Change 483489 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] create static microsite for RT ticket archive, copy static Bugzilla

https://gerrit.wikimedia.org/r/483489

Change 483489 merged by Dzahn:
[operations/puppet@production] create static microsite for RT ticket archive, copy static Bugzilla

https://gerrit.wikimedia.org/r/483489

Change 483604 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] static-rt: add LDAP simple auth

https://gerrit.wikimedia.org/r/483604

Change 483604 merged by Dzahn:
[operations/puppet@production] static-rt: add LDAP simple auth, allow ops

https://gerrit.wikimedia.org/r/483604

Change 484808 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] webserver_static: add proxypass LDAP password for static RT

https://gerrit.wikimedia.org/r/484808

Change 484808 merged by Dzahn:
[operations/puppet@production] webserver_static: add proxypass LDAP password for static RT

https://gerrit.wikimedia.org/r/484808

MoritzMuehlenhoff renamed this task from reinstall RT server with private IP and stretch to reinstall RT server with private IP and Buster.Jul 8 2019, 10:45 AM

Change 543954 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] site: add requesttracker role on moscovium on buster

https://gerrit.wikimedia.org/r/543954

Change 543954 merged by Dzahn:
[operations/puppet@production] site: add requesttracker role on moscovium on buster

https://gerrit.wikimedia.org/r/543954

Change 544031 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] requesttracker: install libapache2-mod-perl2 on buster

https://gerrit.wikimedia.org/r/544031

Change 544031 merged by Dzahn:
[operations/puppet@production] requesttracker: install libapache2-mod-perl2 on buster

https://gerrit.wikimedia.org/r/544031

Change 544069 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] requesttracker: use mod_scgi instead of mod_fastcgi on buster

https://gerrit.wikimedia.org/r/544069

Change 544069 merged by Dzahn:
[operations/puppet@production] requesttracker: use mod_scgi instead of mod_fastcgi on buster

https://gerrit.wikimedia.org/r/544069

Change 534129 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/dns@master] add discovery name for RT, point to moscovium

https://gerrit.wikimedia.org/r/534129

Change 534129 merged by Dzahn:
[operations/dns@master] add discovery name for RT, point to moscovium

https://gerrit.wikimedia.org/r/534129

Change 544077 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] ATS/varnish: replace director for RT with moscovium

https://gerrit.wikimedia.org/r/544077

Change 544078 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] exim: switch mail for RT to moscovium

https://gerrit.wikimedia.org/r/544078

Change 544079 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] mariadb/ferm_misc: allow moscovium to connect instead of ununpentium

https://gerrit.wikimedia.org/r/544079

Change 544080 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] site/install_server: decom ununpentium

https://gerrit.wikimedia.org/r/544080

Change 544079 merged by Dzahn:
[operations/puppet@production] mariadb/ferm_misc: allow moscovium to connect to rt database

https://gerrit.wikimedia.org/r/544079

Change 544078 merged by Dzahn:
[operations/puppet@production] exim: switch mail for RT to moscovium

https://gerrit.wikimedia.org/r/544078

Change 544077 merged by Dzahn:
[operations/puppet@production] ATS/varnish: replace director for RT with moscovium

https://gerrit.wikimedia.org/r/544077

Change 546301 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] ssl: new certificate for RT to contain moscovium.eqiad.wmnet

https://gerrit.wikimedia.org/r/546301

Change 546301 merged by Dzahn:
[operations/puppet@production] ssl: new certificate for RT to contain moscovium.eqiad.wmnet

https://gerrit.wikimedia.org/r/546301

Mentioned in SAL (#wikimedia-operations) [2019-10-25T22:42:30Z] <mutante> moscovium - manually deleting envoy listener on 1443 and letting puppet recreate config because it's not removed if you change the port (T180641)

Mentioned in SAL (#wikimedia-operations) [2019-10-25T22:54:59Z] <mutante> moscovium rm /dev/shm/envoy_shared_memory_0 to revive envoy which failed to run after changing ports and reinstalling it (T180641)

RT seems to be currently pointed at moscovium, and is currently broken: the frontpage doesn't load properly (mixed content messages) and the login doesn't work. If this can't be addressed quickly, perhaps we can rollback to the old host (ununpentium) in the meantime? Thanks!

@faidon I am aware. I'll get it fixed today or revert to the previous one, ack.

Change 546662 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] ssl: re-issue certificate for RT

https://gerrit.wikimedia.org/r/546662

Change 546662 merged by Dzahn:
[operations/puppet@production] ssl: re-issue certificate for RT

https://gerrit.wikimedia.org/r/546662

Mentioned in SAL (#wikimedia-operations) [2019-10-28T18:28:05Z] <mutante> moscovium - deleting /etc/request-tracker4/RT_SiteConfig.d/ 50-debconf.pm and 51-dbconfig-common.pm which duplicate the same files without .pm extension with wrong values, probably due to some package change (T180641)

Mentioned in SAL (#wikimedia-operations) [2019-10-28T18:32:27Z] <mutante> moscovium - rename all files in /etc/request-tracker4/RT_SiteConfig.d to have a .pm extension - this fixed RT - login works again - puppet patch coming up (T180641)

RT seems to be currently pointed at moscovium, and is currently broken: the frontpage doesn't load properly (mixed content messages) and the login doesn't work. If this can't be addressed quickly, perhaps we can rollback to the old host (ununpentium) in the meantime? Thanks!

@faidon RT is fixed Login works again.

Change 546676 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] requesttracker: rename config files to have a .pm extension

https://gerrit.wikimedia.org/r/546676

Change 546676 merged by Dzahn:
[operations/puppet@production] requesttracker: rename config files to have a .pm extension

https://gerrit.wikimedia.org/r/546676

@faidon RT is fixed Login works again.

Thank you!

Change 544080 merged by Dzahn:
[operations/puppet@production] site/install_server: decom ununpentium

https://gerrit.wikimedia.org/r/544080

Change 547051 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] requesttracker: rsync the database dump to new server

https://gerrit.wikimedia.org/r/547051

Change 547052 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] requesttracker: remove rsync and special case for jessie

https://gerrit.wikimedia.org/r/547052

Change 547051 merged by Dzahn:
[operations/puppet@production] requesttracker: rsync the database dump to new server

https://gerrit.wikimedia.org/r/547051

Mentioned in SAL (#wikimedia-operations) [2019-10-29T22:14:18Z] <mutante> rsynced data dump and config from ununpentium to moscovium in /srv/ before shutting down the old server (T180641)

Change 547052 merged by Dzahn:
[operations/puppet@production] requesttracker: remove rsync and special cases for jessie

https://gerrit.wikimedia.org/r/547052

Dzahn closed subtask T236748: decom ununpentium as Resolved.

RT is now on buster on moscovium and using a private IP.

ununpentium the jessie machine has been decom'ed.

Mentioned in SAL (#wikimedia-operations) [2019-10-30T19:05:12Z] <mutante> moscovium - stop and remove rsync server, purge rsync package T180641