Page MenuHomePhabricator
Create Task
Maniphest T219626

codfw1dev: bootstrap cloudcontrol servers in mitaka/stretch
Closed, ResolvedPublic
Actions

Description

Let's install cloudcontrol servers in codfw1dev as mitaka/stretch.

This will also allow us to test the puppet code for these servers, since these will be the first running the mitaka/stretch combo.

Plan is:

  • image cloudcontrol2001-dev with mitaka/stretch (was spare) and test puppet code from T215407: cloudvps: cloudcontrol support for mitaka/stretch
  • pupulate openstack databases, from labtestcontrol2003 and labtestcontrol2001 and put them in cloudcontrol2001-dev
  • reimage and rename labtestcontrol2003 to cloudcontrol2003-dev with mitaka/stretch (currently mitaka/jessie)

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

aborrero created this task.Mar 29 2019, 12:39 PM
aborrero mentioned this in T217891: CloudVPS: rework codfw deployments.
gerritbot added a comment.Mar 29 2019, 1:23 PM

Change 500044 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] wmcs: introduce codfw1dev puppet code

https://gerrit.wikimedia.org/r/500044

gerritbot added a project: Patch-For-Review.Mar 29 2019, 1:23 PM
gerritbot added a comment.Mar 29 2019, 1:41 PM

Change 500044 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] wmcs: introduce codfw1dev puppet code

https://gerrit.wikimedia.org/r/500044

gerritbot added a comment.Mar 29 2019, 1:43 PM

Change 500047 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudcontrol2001-dev: assign proper role

https://gerrit.wikimedia.org/r/500047

gerritbot added a comment.Mar 29 2019, 1:46 PM

Change 500047 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudcontrol2001-dev: assign proper role

https://gerrit.wikimedia.org/r/500047

ops-monitoring-bot added a comment.Apr 1 2019, 10:26 AM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201904011026_aborrero_125072_cloudcontrol2001-dev_wikimedia_org.log.

Stashbot added a comment.Apr 1 2019, 10:27 AM

Mentioned in SAL (#wikimedia-operations) [2019-04-01T10:27:21Z] <arturo> T219626 reimaging cloudcontrol2001-dev

ops-monitoring-bot added a comment.Apr 1 2019, 10:39 AM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia.org']

Of which those FAILED:

['cloudcontrol2001-dev.wikimedia.org']
ops-monitoring-bot added a comment.Apr 1 2019, 10:50 AM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201904011049_aborrero_130582_cloudcontrol2001-dev_wikimedia_org.log.

ops-monitoring-bot added a comment.Apr 1 2019, 11:03 AM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia.org']

Of which those FAILED:

['cloudcontrol2001-dev.wikimedia.org']
gerritbot added a comment.Apr 1 2019, 11:10 AM

Change 500418 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[labs/private@master] openstack: bootstrap hiera keys for codfw1dev

https://gerrit.wikimedia.org/r/500418

gerritbot added a comment.Apr 1 2019, 11:22 AM

Change 500418 merged by Arturo Borrero Gonzalez:
[labs/private@master] openstack: bootstrap hiera keys for codfw1dev

https://gerrit.wikimedia.org/r/500418

ops-monitoring-bot added a comment.Apr 1 2019, 11:25 AM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201904011125_aborrero_140093_cloudcontrol2001-dev_wikimedia_org.log.

ops-monitoring-bot added a comment.Apr 1 2019, 11:38 AM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia.org']

Of which those FAILED:

['cloudcontrol2001-dev.wikimedia.org']
ops-monitoring-bot added a comment.Apr 1 2019, 11:45 AM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201904011145_aborrero_144352_cloudcontrol2001-dev_wikimedia_org.log.

ops-monitoring-bot added a comment.Apr 1 2019, 11:58 AM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia.org']

Of which those FAILED:

['cloudcontrol2001-dev.wikimedia.org']
gerritbot added a comment.Apr 1 2019, 12:23 PM

Change 500434 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] openstack: keystone: add DB configuration for stretch

https://gerrit.wikimedia.org/r/500434

Stashbot added a comment.Apr 1 2019, 12:26 PM

Mentioned in SAL (#wikimedia-cloud) [2019-04-01T12:26:33Z] <arturo> T219626 create VM cloudcontrol-codfw1dev-test-01 for testing codfw1dev cloudcontrol puppet role

gerritbot added a comment.Apr 1 2019, 1:13 PM

Change 500434 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] openstack: add some missing bits for codfw1dev

https://gerrit.wikimedia.org/r/500434

gerritbot added a comment.Apr 1 2019, 3:01 PM

Change 500468 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] hieradata: openstack: codfw1dev: nova: add missing nova keys

https://gerrit.wikimedia.org/r/500468

gerritbot added a comment.Apr 1 2019, 3:08 PM

Change 500468 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] hieradata: openstack: codfw1dev: nova: add missing nova keys

https://gerrit.wikimedia.org/r/500468

ops-monitoring-bot added a comment.Apr 1 2019, 3:10 PM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201904011510_aborrero_192261_cloudcontrol2001-dev_wikimedia_org.log.

ops-monitoring-bot added a comment.Apr 1 2019, 5:02 PM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia.org']

Of which those FAILED:

['cloudcontrol2001-dev.wikimedia.org']
aborrero added a comment.Apr 1 2019, 5:23 PM

Apparently the server was finally installed fine.

gerritbot added a comment.Apr 2 2019, 12:43 PM

Change 500718 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudcontrol2001-dev: fix partman recipe

https://gerrit.wikimedia.org/r/500718

gerritbot added a comment.Apr 2 2019, 12:45 PM

Change 500718 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudcontrol2001-dev: fix partman recipe

https://gerrit.wikimedia.org/r/500718

aborrero added a comment.Apr 2 2019, 12:48 PM

Reimaging it bc wrong partman recipe and because a some changes in the puppet code for mitaka/stretch in cloudcontrol servers.

ops-monitoring-bot added a comment.Apr 2 2019, 12:49 PM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia-org

The log can be found in /var/log/wmf-auto-reimage/201904021249_aborrero_144188_cloudcontrol2001-dev_wikimedia-org.log.

ops-monitoring-bot added a comment.Apr 2 2019, 12:49 PM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia-org']

Of which those FAILED:

['cloudcontrol2001-dev.wikimedia-org']
ops-monitoring-bot added a comment.Apr 2 2019, 12:50 PM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201904021250_aborrero_147374_cloudcontrol2001-dev_wikimedia_org.log.

ops-monitoring-bot added a comment.Apr 2 2019, 1:04 PM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia.org']

Of which those FAILED:

['cloudcontrol2001-dev.wikimedia.org']
gerritbot added a comment.Apr 2 2019, 3:40 PM

Change 500768 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudcontrol2001-dev: use raid1-lvm.cfg partman recipe

https://gerrit.wikimedia.org/r/500768

gerritbot added a comment.Apr 2 2019, 3:42 PM

Change 500768 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudcontrol2001-dev: use raid1-lvm.cfg partman recipe

https://gerrit.wikimedia.org/r/500768

ops-monitoring-bot added a comment.Apr 2 2019, 3:45 PM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201904021545_aborrero_142660_cloudcontrol2001-dev_wikimedia_org.log.

ops-monitoring-bot added a comment.Apr 2 2019, 3:45 PM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia.org']

Of which those FAILED:

['cloudcontrol2001-dev.wikimedia.org']
ops-monitoring-bot added a comment.Apr 2 2019, 3:46 PM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201904021546_aborrero_145046_cloudcontrol2001-dev_wikimedia_org.log.

ops-monitoring-bot added a comment.Apr 2 2019, 3:46 PM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia.org']

Of which those FAILED:

['cloudcontrol2001-dev.wikimedia.org']
ops-monitoring-bot added a comment.Apr 2 2019, 3:46 PM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201904021546_aborrero_147208_cloudcontrol2001-dev_wikimedia_org.log.

Stashbot added a comment.Apr 2 2019, 3:57 PM

Mentioned in SAL (#wikimedia-operations) [2019-04-02T15:57:07Z] <arturo> T219626 reimaging cloudcontrol2001-dev again

ops-monitoring-bot added a comment.Apr 2 2019, 5:56 PM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia.org']

Of which those FAILED:

['cloudcontrol2001-dev.wikimedia.org']
• GTirloni unsubscribed.Apr 3 2019, 10:14 AM
aborrero mentioned this in T219981: openstack: clientpackages puppet entanglement.Apr 3 2019, 12:08 PM
ops-monitoring-bot added a comment.Apr 3 2019, 12:41 PM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201904031241_aborrero_80024_cloudcontrol2001-dev_wikimedia_org.log.

Stashbot added a comment.Apr 3 2019, 12:42 PM

Mentioned in SAL (#wikimedia-operations) [2019-04-03T12:42:29Z] <arturo> T219626 reimaging cloudcontrol2001-dev

ops-monitoring-bot added a comment.Apr 3 2019, 1:17 PM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia.org']

Of which those FAILED:

['cloudcontrol2001-dev.wikimedia.org']
ops-monitoring-bot added a comment.Apr 4 2019, 10:20 AM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201904041020_aborrero_106392_cloudcontrol2001-dev_wikimedia_org.log.

Stashbot added a comment.Apr 4 2019, 10:21 AM

Mentioned in SAL (#wikimedia-operations) [2019-04-04T10:21:27Z] <arturo> T219626 reimaging cloudcontrol2001-dev again

ops-monitoring-bot added a comment.Apr 4 2019, 10:57 AM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia.org']

Of which those FAILED:

['cloudcontrol2001-dev.wikimedia.org']
Stashbot added a comment.Apr 4 2019, 11:08 AM

Mentioned in SAL (#wikimedia-operations) [2019-04-04T11:08:09Z] <arturo> drop python-psutil from jessie-wikimedia/openstack-mitaka-jessie, related to T219626

ops-monitoring-bot added a comment.Apr 4 2019, 11:20 AM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201904041120_aborrero_120318_cloudcontrol2001-dev_wikimedia_org.log.

Stashbot added a comment.Apr 4 2019, 11:21 AM

Mentioned in SAL (#wikimedia-operations) [2019-04-04T11:21:18Z] <arturo> T219626 reimaging cloudcontrol2001-dev again

gerritbot added a comment.Apr 4 2019, 11:21 AM

Change 501180 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] aptrepo: drop gdb from openstack-mitaka-jessie

https://gerrit.wikimedia.org/r/501180

gerritbot added a comment.Apr 4 2019, 11:22 AM

Change 501180 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] aptrepo: drop gdb from openstack-mitaka-jessie

https://gerrit.wikimedia.org/r/501180

gerritbot added a comment.Apr 4 2019, 11:27 AM

Change 501181 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudcontrol2001-dev: add IPv6

https://gerrit.wikimedia.org/r/501181

gerritbot added a comment.Apr 4 2019, 11:28 AM

Change 501181 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudcontrol2001-dev: add IPv6

https://gerrit.wikimedia.org/r/501181

ops-monitoring-bot added a comment.Apr 4 2019, 11:46 AM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia.org']

and were ALL successful.

gerritbot added a comment.Apr 4 2019, 11:52 AM

Change 501193 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/dns@master] cloudcontrol2001-dev: add IPv6

https://gerrit.wikimedia.org/r/501193

gerritbot added a comment.Apr 4 2019, 12:00 PM

Change 501193 merged by Arturo Borrero Gonzalez:
[operations/dns@master] cloudcontrol2001-dev: add IPv6

https://gerrit.wikimedia.org/r/501193

ops-monitoring-bot added a comment.Apr 4 2019, 12:10 PM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201904041210_aborrero_132473_cloudcontrol2001-dev_wikimedia_org.log.

Stashbot added a comment.Apr 4 2019, 12:10 PM

Mentioned in SAL (#wikimedia-operations) [2019-04-04T12:10:47Z] <arturo> T219626 reimaging cloudcontrol2001-dev again

aborrero mentioned this in T218569: Openstack codfw DBs: move to m5-master.eqiad.wmnet.Apr 4 2019, 12:20 PM
aborrero updated the task description. (Show Details)
aborrero updated the task description. (Show Details)
ops-monitoring-bot added a comment.Apr 4 2019, 12:36 PM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia.org']

and were ALL successful.

aborrero added a subtask: T220173: cloudcontrol2001-dev hourly cronspam.Apr 5 2019, 9:14 AM
aborrero mentioned this in T220173: cloudcontrol2001-dev hourly cronspam.
aborrero closed subtask T220173: cloudcontrol2001-dev hourly cronspam as Resolved.Apr 5 2019, 1:04 PM
ops-monitoring-bot added a comment.Apr 12 2019, 10:32 AM

Script wmf-auto-reimage was launched by aborrero on cumin1001.eqiad.wmnet for hosts:

cloudcontrol2001-dev.wikimedia.org

The log can be found in /var/log/wmf-auto-reimage/201904121032_aborrero_185525_cloudcontrol2001-dev_wikimedia_org.log.

Stashbot added a comment.Apr 12 2019, 10:32 AM

Mentioned in SAL (#wikimedia-operations) [2019-04-12T10:32:51Z] <arturo> T219626 reimaging cloudcontrol2001-dev

aborrero closed subtask T220096: codfw1dev: decide which DBs to reallocate to clouddb2001-dev as Resolved.Apr 12 2019, 10:49 AM
ops-monitoring-bot added a comment.Apr 12 2019, 10:55 AM

Completed auto-reimage of hosts:

['cloudcontrol2001-dev.wikimedia.org']

Of which those FAILED:

['cloudcontrol2001-dev.wikimedia.org']
gerritbot added a comment.Apr 12 2019, 12:00 PM

Change 503327 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloudcontrol2001-dev: drop local keystone db

https://gerrit.wikimedia.org/r/503327

gerritbot added a comment.Apr 12 2019, 12:05 PM

Change 503327 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloudcontrol2001-dev: drop local keystone db

https://gerrit.wikimedia.org/r/503327

Stashbot added a comment.Apr 15 2019, 1:08 PM

Mentioned in SAL (#wikimedia-cloud) [2019-04-15T13:08:35Z] <arturo> T219626 add DB grants for keystone/nova/nova_api to clouddb2001-dev (codfw1dev)

Stashbot added a comment.Apr 16 2019, 10:30 AM

Mentioned in SAL (#wikimedia-cloud) [2019-04-16T10:29:52Z] <arturo> T219626 replace labtestcontrol2003 with cloudcontrol2001-dev in the clouddb2001-dev database (codfw1dev deployment)

Stashbot added a comment.Apr 16 2019, 10:37 AM

Mentioned in SAL (#wikimedia-cloud) [2019-04-16T10:37:00Z] <arturo> T219626 replace 208.80.153.75 with 208.80.153.59 in the clouddb2001-dev database (codfw1dev deployment)

gerritbot added a comment.Apr 16 2019, 11:15 AM

Change 504298 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] openstack: codfw1dev: update DB for neutron-server

https://gerrit.wikimedia.org/r/504298

gerritbot added a comment.Apr 16 2019, 11:25 AM

Change 504298 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] openstack: codfw1dev: update DB for neutron-server

https://gerrit.wikimedia.org/r/504298

Stashbot added a comment.Apr 16 2019, 11:27 AM

Mentioned in SAL (#wikimedia-cloud) [2019-04-16T11:27:41Z] <arturo> T219626 add DB grants for neutron and glnace to clouddb2001-dev (codfw1dev)

aborrero closed this task as Resolved.Apr 16 2019, 11:32 AM
aborrero updated the task description. (Show Details)

The bootstrap should be now complete. Will open separate tasks for non-working stuff we may discover.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits