Page MenuHomePhabricator

upgrade and rename krypton & create its codfw equivalent
Closed, ResolvedPublic

Description

The server krypton.eqiad.wmnet is a "webserver_misc_apps".

Things we should probably do:

  • upgrade from jessie to stretch / switch to PHP 7
  • create an equivalent of it in codfw for failover | miscweb2001 is up
  • pick a "cluster" name for them instead of a misc host name and rename them to something1001/2001 | it is called "miscweb", created on wiki and in puppet

services hosted here that are moving to miscweb1001:

Details

ProjectBranchLines +/-Subject
operations/puppetproduction+2 -1
operations/puppetproduction+1 -0
operations/puppetproduction+1 -0
operations/puppetproduction+4 -2
operations/puppetproduction+5 -2
operations/puppetproduction+5 -2
operations/puppetproduction+7 -8
operations/puppetproduction+2 -6
operations/puppetproduction+1 -6
operations/puppetproduction+3 -3
operations/puppetproduction+3 -0
operations/puppetproduction+2 -7
operations/puppetproduction+2 -8
operations/puppetproduction+3 -3
operations/puppetproduction+13 -12
operations/puppetproduction+6 -3
operations/puppetproduction+2 -0
operations/puppetproduction+9 -1
operations/puppetproduction+1 -1
operations/puppetproduction+2 -0
operations/puppetproduction+1 -1
operations/puppetproduction+1 -1
operations/puppetproduction+5 -0
operations/dnsmaster+2 -0
operations/puppetproduction+1 -1
operations/puppetproduction+1 -0
operations/puppetproduction+1 -1
Show related patches Customize query in gerrit

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 512446 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/dns@master] rename krypton to miscweb1001

https://gerrit.wikimedia.org/r/512446

Change 513215 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] netboot: add miscweb[12]00[12] to partman

https://gerrit.wikimedia.org/r/513215

Change 513215 merged by Dzahn:
[operations/puppet@production] netboot: add miscweb[12]00[12] to partman

https://gerrit.wikimedia.org/r/513215

Change 513227 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] cumin: add alias for new miscweb* cluster name

https://gerrit.wikimedia.org/r/513227

Change 513230 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] add miscweb2001 to role webserver_misc_apps in site.pp

https://gerrit.wikimedia.org/r/513230

Change 513227 merged by Dzahn:
[operations/puppet@production] cumin: add alias for new miscweb* cluster name

https://gerrit.wikimedia.org/r/513227

Change 513230 merged by Dzahn:
[operations/puppet@production] add miscweb2001 to role webserver_misc_apps in site.pp

https://gerrit.wikimedia.org/r/513230

Dzahn raised the priority of this task from Medium to High.Aug 27 2019, 9:41 AM

Change 512446 merged by Dzahn:
[operations/dns@master] add IP for miscweb1001

https://gerrit.wikimedia.org/r/512446

Change 532683 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] site: add miscweb role to miscweb1001

https://gerrit.wikimedia.org/r/532683

Dzahn updated the task description. (Show Details)Aug 27 2019, 10:17 AM

Mentioned in SAL (#wikimedia-operations) [2019-08-27T10:25:05Z] <mutante> ganeti eqiad - creating new VM with same specs as krypton to replace it with a stretch instance and mirror miscweb2001. krypton to be removed (T224323, T105507, T224247)

Change 532687 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] install_server: add miscweb1001 to DHCP

https://gerrit.wikimedia.org/r/532687

Change 532687 merged by Dzahn:
[operations/puppet@production] install_server: add miscweb1001 to DHCP

https://gerrit.wikimedia.org/r/532687

Change 532683 merged by Dzahn:
[operations/puppet@production] site: add miscweb role to miscweb1001

https://gerrit.wikimedia.org/r/532683

Mentioned in SAL (#wikimedia-operations) [2019-08-27T11:51:31Z] <mutante> miscweb1001 - manually remove tin.eqiad.wmnet (!) from /srv/iegreview/iegreview-cache/.config and replace with deploy1001 after first puppet run. still existing bug that tin is not fully removed (T224247, T175288, T197470)

Mentioned in SAL (#wikimedia-operations) [2019-08-27T11:54:56Z] <mutante> miscweb1001 - a2dismod mpm_event ; a2enmod php7.0 ; systemctl restart apache2 (T224247, T196968) please also see https://gerrit.wikimedia.org/r/c/operations/puppet/+/451206

Change 532695 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] trafficserver/varnish: replace krypton with miscweb1001, rename director

https://gerrit.wikimedia.org/r/532695

Change 532701 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] site/install_server: remote krypton.eqiad.wmnet

https://gerrit.wikimedia.org/r/532701

Change 532702 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] logstash: replace krypton with grafana1001 in collector ferm rules

https://gerrit.wikimedia.org/r/532702

Change 532702 merged by Dzahn:
[operations/puppet@production] logstash: replace krypton with grafana1001 in collector ferm rules

https://gerrit.wikimedia.org/r/532702

Change 532862 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] install_server: switch miscweb servers to buster installer

https://gerrit.wikimedia.org/r/532862

Change 532862 merged by Dzahn:
[operations/puppet@production] install_server: switch miscweb servers to buster installer

https://gerrit.wikimedia.org/r/532862

Change 532948 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] misc_apps::httpd: do not load SSL httpd module

https://gerrit.wikimedia.org/r/532948

Change 532948 merged by Dzahn:
[operations/puppet@production] misc_apps::httpd: do not load SSL httpd module

https://gerrit.wikimedia.org/r/532948

Change 532954 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] misc_apps::httpd: allow port 80 from deployment servers

https://gerrit.wikimedia.org/r/532954

Change 532954 merged by Dzahn:
[operations/puppet@production] misc_apps::httpd: allow port 80 from deployment servers

https://gerrit.wikimedia.org/r/532954

Change 532959 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] iegreview: require a mysql client to be installed

https://gerrit.wikimedia.org/r/532959

Change 532959 merged by Dzahn:
[operations/puppet@production] iegreview: require a mysql client to be installed

https://gerrit.wikimedia.org/r/532959

Change 533024 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] ATS/varnish: add director miscweb and switch racktables to it

https://gerrit.wikimedia.org/r/533024

Change 533024 merged by Dzahn:
[operations/puppet@production] ATS/varnish: add director miscweb and switch racktables to it

https://gerrit.wikimedia.org/r/533024

Change 533154 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] ATS/varnish: switch iegreview to miscweb backend and use TLS

https://gerrit.wikimedia.org/r/533154

Change 533155 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] racktables: remove jessie support

https://gerrit.wikimedia.org/r/533155

Change 533157 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] iegreview: remove jessie support

https://gerrit.wikimedia.org/r/533157

Change 533158 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] wikimania_scholarships: remove jessie/php5 support

https://gerrit.wikimedia.org/r/533158

Change 532695 abandoned by Dzahn:
ATS/varnish: replace krypton with miscweb1001, rename director

Reason:
doing in separate changes instead (https://gerrit.wikimedia.org/r/c/operations/puppet/ /533024, https://gerrit.wikimedia.org/r/c/operations/puppet/ /533154 )

https://gerrit.wikimedia.org/r/532695

Change 533159 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] misc_apps::httpd: remove jessie/php5 support

https://gerrit.wikimedia.org/r/533159

Change 533154 merged by Dzahn:
[operations/puppet@production] ATS/varnish: switch iegreview to miscweb backend and use TLS

https://gerrit.wikimedia.org/r/533154

Change 533155 merged by Dzahn:
[operations/puppet@production] racktables: remove jessie support

https://gerrit.wikimedia.org/r/533155

Change 533157 merged by Dzahn:
[operations/puppet@production] iegreview: remove jessie support

https://gerrit.wikimedia.org/r/533157

Change 533175 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] ATS/varnish: switch scholarschips to miscweb and use TLS

https://gerrit.wikimedia.org/r/533175

Change 533181 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] ATS: configure "never_cache" for miscweb1001 backend

https://gerrit.wikimedia.org/r/533181

Change 533181 merged by Dzahn:
[operations/puppet@production] ATS: configure "never-cache" for webserver-misc-apps.discovery.wmnet

https://gerrit.wikimedia.org/r/533181

Change 533175 merged by Dzahn:
[operations/puppet@production] ATS/varnish: switch wikimania scholarships to miscweb, use TLS

https://gerrit.wikimedia.org/r/533175

Change 533158 merged by Dzahn:
[operations/puppet@production] wikimania_scholarships: remove jessie/php5 support

https://gerrit.wikimedia.org/r/533158

Change 533159 merged by Dzahn:
[operations/puppet@production] misc_apps::httpd: remove jessie/php5 support

https://gerrit.wikimedia.org/r/533159

Change 532701 merged by Dzahn:
[operations/puppet@production] site/install_server: decom krypton.eqiad.wmnet

https://gerrit.wikimedia.org/r/532701

Dzahn updated the task description. (Show Details)Aug 29 2019, 12:22 PM
Dzahn removed a project: Patch-For-Review.
Dzahn updated the task description. (Show Details)

The following services have been moved away from krypton miscweb1001.

  • new DNS discovery record 'webserver-misc-apps.discovery.wmnet is an alias for miscweb1001.eqiad.wmnet
  • miscweb1001.eqiad.wmnet is on stretch (because we don't have the envoy package for buster just yet but coming soon)
  • using PHP7 instead of PHP5, PHP5 support in puppet code removed
  • using envoy for TLS termination on backend, https:// url in ATS config for this backend
  • equivalent of backend server in codfw exists as miscweb2001.codfw.wmnet and also the DBA connection should not be an issue anymore, though untested

TODO: DB config needs to use codfw proxy if puppet role applied on codfw node

Joe moved this task from Incoming 🐫 to Doing 😎 on the serviceops board.Sep 16 2019, 3:02 PM

krypton has been decom'ed fully

Change 537761 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] racktables: set db host in Hiera, set to eqiad

https://gerrit.wikimedia.org/r/537761

Change 537762 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] iegreview: set db host in Hiera

https://gerrit.wikimedia.org/r/537762

Change 537763 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] wikimania_scholarships: set db host in Hiera

https://gerrit.wikimedia.org/r/537763

Change 537762 merged by Dzahn:
[operations/puppet@production] iegreview: set db host in Hiera

https://gerrit.wikimedia.org/r/537762

Change 537763 merged by Dzahn:
[operations/puppet@production] wikimania_scholarships: set db host in Hiera

https://gerrit.wikimedia.org/r/537763

Change 537761 merged by Dzahn:
[operations/puppet@production] racktables: set db host in Hiera, set to eqiad, use lookup

https://gerrit.wikimedia.org/r/537761

hashar removed a subscriber: hashar.Oct 25 2019, 11:52 AM
akosiaris closed this task as Resolved.Nov 20 2019, 8:31 AM
akosiaris added a subscriber: akosiaris.

krypton is no more since 7a36b4e7a94f486a400f0363c263c446c33bba80, resolving.

Dzahn reopened this task as Open.Nov 20 2019, 1:19 PM
Dzahn added a comment.Nov 22 2019, 5:51 PM

That's true, just had one last little todo here for the one in codfw. Doing that now.

Change 552551 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] wikimania_scholarships app: use codfw database when in codfw

https://gerrit.wikimedia.org/r/552551

Change 552552 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] iegreview app: use codfw database when in codfw

https://gerrit.wikimedia.org/r/552552

Change 552553 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] racktables: use codfw database when in codfw

https://gerrit.wikimedia.org/r/552553

Change 552551 merged by Dzahn:
[operations/puppet@production] wikimania_scholarships app: use codfw database when in codfw

https://gerrit.wikimedia.org/r/552551

Change 552552 merged by Dzahn:
[operations/puppet@production] iegreview app: use codfw database when in codfw

https://gerrit.wikimedia.org/r/552552

Change 552553 merged by Dzahn:
[operations/puppet@production] racktables: use codfw database when in codfw

https://gerrit.wikimedia.org/r/552553

Dzahn closed this task as Resolved.Nov 25 2019, 10:45 PM
Dzahn removed a project: Patch-For-Review.