⚓ T97163 Support instance creation/deletion via nova commandline

Subject	Repo	Branch	Lines +/-
Remove ldap host-entry creation and deletion from OSM.	mediawiki/extensions/OpenStackManager	wmf/1.26wmf9	+6 -189
Use fqdn instead of ecid for ldap host dn	mediawiki/extensions/OpenStackManager	wmf/1.26wmf9	+56 -76
Remove ldap host-entry creation and deletion from OSM.	mediawiki/extensions/OpenStackManager	master	+6 -189
Use fqdn instead of ecid for ldap host dn	mediawiki/extensions/OpenStackManager	master	+56 -76
Remove two maintenance scripts.	mediawiki/extensions/OpenStackManager	master	+0 -305

Status	Assigned	Task
Resolved	Andrew	T42525 Cant add a security group to an existing instance
Resolved	Andrew	T87279 Make OpenStack Horizon useful for production labs
Resolved	hashar	T96670 Instances created by Nodepool cant run puppet due to missing certificate
Resolved	Andrew	T97163 Support instance creation/deletion via nova commandline
Resolved	Andrew	T91987 Nova Instance creation hook for ldap
Duplicate	Andrew	T95910 Abolish use of ec2 ids
Resolved	Andrew	T95911 Remove puppet and salt keys on instance deletion
Resolved	Andrew	T95480 Abolish use of ec2id in cert names
Resolved	ArielGlenn	T95481 Fix monitor_labs_salt_keys.py to handle the new labs naming scheme
Resolved	Andrew	T95519 Automatically clean salt and puppet certs on instance deletion
Resolved	Andrew	T97170 Switch all of labs to the pdns server, remove the use_dnsmasq flag.
Resolved	coren	T95288 Designate should support split horizon resolution to yield private IP of instances behind a public DNS entry
Resolved	Andrew	T99133 New server for labs dns recursor
Resolved	Andrew	T101871 remove per-host sudo control from OSM
Resolved	Andrew	T102047 Rename all labs ldap host entries to use fqdn
Resolved	Andrew	T102834 instance status pages broken
Resolved	Andrew	T102839 Build new images that don't do an ec2 id lookup in ldap

Andrew created this task.Apr 24 2015, 7:53 PM

Andrew claimed this task.

Andrew raised the priority of this task from to Needs Triage.

Andrew updated the task description. (Show Details)

Andrew added a project: Cloud-Services.

Andrew subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 24 2015, 7:53 PM

Andrew triaged this task as High priority.Apr 24 2015, 7:58 PM

Andrew set Security to None.

Andrew added subtasks: T91987: Nova Instance creation hook for ldap, T95480: Abolish use of ec2id in cert names.

Right now, new instances get two ldap records. The first is from OSM:

i-00000b91.eqiad.wmflabs, hosts, wikimedia.org

dn: dc=i-00000b91.eqiad.wmflabs,ou=hosts,dc=wikimedia,dc=org
objectClass: domainrelatedobject
objectClass: dnsdomain
objectClass: puppetclient
objectClass: domain
objectClass: dcobject
objectClass: top
puppetVar: use_dnsmasq=true
puppetVar: realm=labs
puppetVar: instanceproject=integration
puppetVar: instancename=integration-saltmaster
puppetClass: base
puppetClass: role::labs::instance
puppetClass: puppetmaster::autosigner
puppetClass: role::salt::masters::labs::project_master
l: eqiad
associatedDomain: i-00000b91.eqiad.wmflabs
associatedDomain: integration-saltmaster.eqiad.wmflabs
associatedDomain: i-00000b91.integration.eqiad.wmflabs
associatedDomain: integration-saltmaster.integration.eqiad.wmflabs
dc: i-00000b91.eqiad.wmflabs
aRecord: 10.68.18.24

The second is from a sink callback:

integration-saltmaster.integration.eqiad.wmflabs, hosts, wikimedia.org

dn: dc=integration-saltmaster.integration.eqiad.wmflabs,ou=hosts,dc=wikimedia,
dc=org
objectClass: domainrelatedobject
objectClass: dnsdomain
objectClass: puppetclient
objectClass: domain
objectClass: dcobject
objectClass: top
puppetVar: realm=labs
puppetVar: use_dnsmasq=true
puppetVar: instanceproject=integration
puppetVar: instancename=integration-saltmaster
puppetClass: base
puppetClass: role::labs::instance
l: eqiad
associatedDomain: TESTING-integration-saltmaster.integration.eqiad.wmflabs
associatedDomain: TESTING-integration-saltmaster.eqiad.wmflabs
dc: integration-saltmaster.integration.eqiad.wmflabs
aRecord: 10.68.18.24

What will it take for us to only use the latter?

In no particular order:

Change OSM editing feature so that it edits fqdn-style records rather than ec2id-style records
Rename all ec2-style records to fqdn-style records
Change sink callback to create live fqdn records rather than dummy records
Purge existing dummy records
Change puppet cert so it matches the new records: https://gerrit.wikimedia.org/r/#/c/202924/
Make sure puppet certs for deleted instances are deleted: https://gerrit.wikimedia.org/r/#/c/205897/

In order to switch to the new cert names, it's nice to have a consistent fqdn. So... can we make that depend on 'move all instances to new dns server'?

hashar added a parent task: T96670: Instances created by Nodepool cant run puppet due to missing certificate.Apr 24 2015, 8:22 PM

hashar subscribed.

Andrew added a project: Labs-Sprint-101.Jun 8 2015, 5:32 PM

Andrew closed subtask T97170: Switch all of labs to the pdns server, remove the use_dnsmasq flag. as Resolved.Jun 8 2015, 8:15 PM

Andrew added a subtask: T101871: remove per-host sudo control from OSM.Jun 10 2015, 7:13 PM

Change 217039 had a related patch set uploaded (by Andrew Bogott):
Use fqdn instead of ecid for ldap host dn

https://gerrit.wikimedia.org/r/217039

Change 217345 had a related patch set uploaded (by Andrew Bogott):
Remove two maintenance scripts.

https://gerrit.wikimedia.org/r/217345

Change 217416 had a related patch set uploaded (by Andrew Bogott):
Remove ldap host-entry creation and deletion from OSM.

https://gerrit.wikimedia.org/r/217416

Change 217345 merged by jenkins-bot:
Remove two maintenance scripts.