On beta, the Swift cluster is using Ubuntu Trusty and should be migrated to Debian Jessie. The three instances are:
| Instance | IP | Flavor |
|---|---|---|
| deployment-ms-fe01 | 10.68.16.96 | m1.small |
| deployment-ms-be01 | 10.68.16.24 | m1.xlarge |
| deployment-ms-be02 | 10.68.21.135 | m1.xlarge |
Note the backend instances use m1.xlarge flavor (8 CPUs, 16G RAM and 160G disk). Can probably switch to m1.large instead (4 CPUs, 8G RAM and 80G disk) which should fit.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | hashar | T160990 deployment-ms-be03.deployment-prep and deployment-ms-be04.deployment-prep have high load / system CPU | |||
| Resolved | fgiunchedi | T162247 Migrate beta cluster Swift cluster from Trusty to Jessie |
Indeed we'll have to do this also because production will no longer have trusty "soon" (cfr T162609). I'll start with provisioning a jessie ms-fe since that's the easiest and will allow us to test swift 2.10 too.
@hashar for ms-be the used ram seems in the order of ~12GB so m1.large would be tight. I'll go with m1.xlarge for now, we can revisit if resources get tighter
Change 350389 had a related patch set uploaded (by Filippo Giunchedi):
[operations/puppet@production] swift: create required LV in labs
ms-be0[34] and ms-fe02 are up and running with swift 2.10, next steps:
Mentioned in SAL (#wikimedia-releng) [2017-04-26T15:24:40Z] <godog> add new deployment-ms-be0[34] backends to swift in deployment-prep - T162247
For reference, switching mw to talk to deployment-ms-fe02 the configuration is here: https://horizon.wikimedia.org/project/prefixpuppet/?tab=prefix_puppet__puppet-deployment-cache-upload for the varnish bits and here https://wikitech.wikimedia.org/wiki/Hiera:Deployment-prep for swift
Mentioned in SAL (#wikimedia-releng) [2017-05-02T09:41:37Z] <godog> flip deployment-cache-upload04 to deployment-ms-fe02 - T162247
Also private/PrivateSettings.php needed a change on deployment-tin to pick up deployment-ms02
Change 351285 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] swift: default to 127.0.0.1 for memcached
I'm going to pause (but not terminate) old deployment-ms trusty hosts since they should be completely decomissioned now. Will terminate say next week if no issues come up.
Change 351285 merged by Filippo Giunchedi:
[operations/puppet@production] swift: default to 127.0.0.1 for memcached
Change 350389 merged by Filippo Giunchedi:
[operations/puppet@production] profile: introduce swift::storage::labs