Page MenuHomePhabricator

upgrade salt in labs
Closed, ResolvedPublic


Upgrade salt from 2014.1.11 to 2014.7.5 on labs instances

Event Timeline

ArielGlenn claimed this task.
ArielGlenn raised the priority of this task from to Normal.
ArielGlenn updated the task description. (Show Details)
ArielGlenn added a subscriber: ArielGlenn.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 8 2015, 7:44 AM
ArielGlenn closed this task as Resolved.May 8 2015, 8:04 AM

This has been done. Some instances were skipped, a full list follows.

  1. Instances that were shut off at the time:

Instance: i-000000fd Status: SHUTOFF hostname: mwreview-merl
Instance: i-0000016c Status: SHUTOFF hostname: basictestsugar
Instance: i-0000016f Status: SHUTOFF hostname: jitsu
Instance: i-00000171 Status: SHUTOFF hostname: labs-vmbuilder-precise
Instance: i-00000175 Status: SHUTOFF hostname: labs-vmbuilder-lucid
Instance: i-00000180 Status: SHUTOFF hostname: pirsquared-test
Instance: i-00000198 Status: SHUTOFF hostname: contributors-metrics
Instance: i-000001af Status: SHUTOFF hostname: gwtoolset
Instance: i-000001cf Status: SHUTOFF hostname: nginx-dev1
Instance: i-000001d6 Status: SHUTOFF hostname: openid-wiki
Instance: i-000001d7 Status: SHUTOFF hostname: openid-wiki2
Instance: i-00000232 Status: SHUTOFF hostname: pdbhandler-1
Instance: i-00000260 Status: SHUTOFF hostname: language-replag-master
Instance: i-00000261 Status: SHUTOFF hostname: language-replag-slave
Instance: i-00000262 Status: SHUTOFF hostname: language-replag-wiki
Instance: i-00000265 Status: SHUTOFF hostname: language-browsertests
Instance: i-00000267 Status: SHUTOFF hostname: language-testlink
Instance: i-0000029e Status: SHUTOFF hostname: parsoid-spof
Instance: i-000002b2 Status: SHUTOFF hostname: maps-warper
Instance: i-000003ab Status: SHUTOFF hostname: parsoid-square
Instance: i-0000065e Status: SHUTOFF hostname: phab-test
Instance: i-000006ce Status: SHUTOFF hostname: filippo-test-precise
Instance: i-000007d7 Status: SHUTOFF hostname: filippo-m3-c2
Instance: i-0000080a Status: SHUTOFF hostname: educationdashboard2
Instance: i-0000086a Status: SHUTOFF hostname: fab
Instance: i-00000952 Status: SHUTOFF hostname: educationdashboard-i18n

  1. Instances that are part of a mini-cluster running their own salt master

I won't list those, you know if you do this or not.
There is one exception; I did mistakenly upgrade services-deploy. Please contact me if that is an issue.

  1. Instances that were responsive to the virt 1000 salt master and did not get updated:

MySQL package issue {'i-000002b5.eqiad.wmflabs': '2014.1.11'}
python-requests package issue {'i-00000122.eqiad.wmflabs': '2014.1.11'}
Linux Headers package issue {'i-0000014b.eqiad.wmflabs': '2014.1.11'}
IO errors/fs errors {'i-000007a1.eqiad.wmflabs': '0.17.5'}

  1. Instances not responsive to salt-master and not upgraded:

MySql package issue: Instance: i-00000325 hostname: wikimetrics-dev1
denied ssh in (public key): Instance: i-000007c5 hostname: sol
denied ssh in (public key): Instance: i-00000b8d hostname: trusty-medium-1429649189
failed dns lookup for hostname: Instance: i-00000363 hostname: incident-test


Other instances that were not responsive to salt-master were upgraded manually via ssh. I did not try to determine why they could not communicate with the salt master. I guess ferm rules but the owners of the instances can check. Those were the following:

Instance: i-000000ee hostname: util-abogott
Instance: i-00000114 hostname: bastion3
Instance: i-00000585 hostname: language-dev
Instance: i-000007e7 hostname: moses1
Instance: i-000009b0 hostname: wmt-bastion
Instance: i-000009b1 hostname: wmt-exec
Instance: i-000009b2 hostname: wmt-apache
Instance: i-000009f6 hostname: bastion-restricted-pdns
Instance: i-00000a06 hostname: sentry-packaging
Instance: i-00000a50 hostname: bastion-pdns
Instance: i-00000c25 hostname: etcd01
Instance: i-00000c26 hostname: etcd02

ArielGlenn set Security to None.May 8 2015, 8:05 AM
ArielGlenn added subscribers: yuvipanda, Andrew.