Page MenuHomePhabricator

CloudVPS: codfw1dev: missing bits
Closed, ResolvedPublic

Description

This task is to track missing bits related to the codfw1dev CloudVPS deployment https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Deployments#codfw
The goal is to have a deployment "mostly" usable that serves as a place to test more stuff related to our openstack setup.

  • DNS. There is no DNS support for VMs in this deployment. Ideally we would introduce the new naming scheme https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/DNS_domain_usage#Resolution
  • Bastions. We don't have a setup for jumping using SSH as we do in eqiad1. Having this setup in place would be really interesting for testing stuff inside VMs (although we can use the virsh console anyway)
  • puppetmaster, cumin, etc
  • LDAP. There are some missing bits in the LDAP setup. I don't even know which is missing, but last time I checked they were no LDAP users in VMs in codfw1dev.
  • Glance images. We don't have same glance images as we have in eqiad1. We have some glance images hardcoding a root password (which nobody knows) which is something to avoid. This is probably related to the LDAP thing. -- No production glance images work fine in codfw1dev.
  • Horizon. Long time ago we used to have a testing horizon setup for codfw. Now we don't have one. This is more optional that the other bullet points.

Related Objects

StatusSubtypeAssignedTask
Resolvedaborrero
ResolvedNone
ResolvedAndrew
ResolvedAndrew
ResolvedCDanis
ResolvedMarostegui
ResolvedAndrew
Resolvedaborrero
InvalidNone
Resolvedaborrero
ResolvedKrenair
Resolvedaborrero
Resolvedjcrespo
ResolvedAndrew
ResolvedNone
Resolvedaborrero
Duplicatejbond
ResolvedAndrew
ResolvedPapaul
ResolvedAndrew
InvalidNone
ResolvedPapaul
ResolvedAndrew
Resolvedayounsi

Event Timeline

Change 534657 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] codf1dev: move the puppetmaster enc database to cloudb2001-dev

https://gerrit.wikimedia.org/r/534657

Change 534657 merged by Andrew Bogott:
[operations/puppet@production] codf1dev: move the puppetmaster enc database to cloudb2001-dev

https://gerrit.wikimedia.org/r/534657

Change 534850 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] codfw1dev: disable the mwyaml backend

https://gerrit.wikimedia.org/r/534850

Change 534850 merged by Andrew Bogott:
[operations/puppet@production] codfw1dev: disable the mwyaml backend

https://gerrit.wikimedia.org/r/534850

Change 534851 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] codfw1dev: update labtest.hiera.yaml to use codfw1dev resources

https://gerrit.wikimedia.org/r/534851

Change 534851 merged by Andrew Bogott:
[operations/puppet@production] codfw1dev: update labtest.hiera.yaml to use codfw1dev resources

https://gerrit.wikimedia.org/r/534851

Change 534938 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] realm.pp: Update certname validation for cloud VMs

https://gerrit.wikimedia.org/r/534938

Change 534938 merged by Andrew Bogott:
[operations/puppet@production] realm.pp: Update certname validation for cloud VMs

https://gerrit.wikimedia.org/r/534938

I have instance designate/dns working now on codfw1dev with the .cloud tld:

root@testinstance-18:~# hostname -f
testinstance-18.devtest.codfw1dev.cloud
root@testinstance-18:~# dig +short testinstance-18.devtest.codfw1dev.cloud
172.16.128.21
root@testinstance-18:~# dig +short -x 172.16.128.21
testinstance-18.devtest.codfw1dev.cloud.

Change 536672 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] codfw1dev: First pass at buildingout cloudweb2001-dev

https://gerrit.wikimedia.org/r/536672

Change 536672 merged by Andrew Bogott:
[operations/puppet@production] codfw1dev: First pass at building out cloudweb2001-dev

https://gerrit.wikimedia.org/r/536672

Change 537127 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] cloudweb2001-dev: remove wikitech profiles

https://gerrit.wikimedia.org/r/537127

Change 537127 merged by Andrew Bogott:
[operations/puppet@production] cloudweb2001-dev: remove wikitech profiles

https://gerrit.wikimedia.org/r/537127

I have instance designate/dns working now on codfw1dev with the .cloud tld:

root@testinstance-18:~# hostname -f
testinstance-18.devtest.codfw1dev.cloud
root@testinstance-18:~# dig +short testinstance-18.devtest.codfw1dev.cloud
172.16.128.21
root@testinstance-18:~# dig +short -x 172.16.128.21
testinstance-18.devtest.codfw1dev.cloud.

wow! awesome work, thanks @Andrew !!

regarding ldap: I just created a new project in codfw1dev and added a member. Ldap config looks correct to me, for example:

271 cn=project-keystonehooktest1,ou=groups,dc=wikimedia,dc=org
objectClass: groupOfNames
objectClass: posixGroup
objectClass: top
gidNumber: 52909
cn: project-keystonehooktest1
member: uid=novaadmin,ou=people,dc=wikimedia,dc=org
member: uid=labtestandrew,ou=people,dc=wikimedia,dc=org

I've no doubt this was broken in the past, but going to mark as resolved for now pending a specific issue.

Change 543943 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/mediawiki-config@master] labtestwikitech: use the new codfw1-dev servers

https://gerrit.wikimedia.org/r/543943

Change 543943 merged by jenkins-bot:
[operations/mediawiki-config@master] labtestwikitech: use the new codfw1-dev servers

https://gerrit.wikimedia.org/r/543943

Andrew moved this task from Doing to Inbox on the cloud-services-team (Kanban) board.

Change 565431 had a related patch set uploaded (by Alex Monk; owner: Alex Monk):
[operations/puppet@production] CloudVPS: codfw1dev: Fix default SSH rule to use correct range

https://gerrit.wikimedia.org/r/565431

Change 565431 merged by Andrew Bogott:
[operations/puppet@production] CloudVPS: codfw1dev: Fix default SSH rule to use correct range

https://gerrit.wikimedia.org/r/565431

Mentioned in SAL (#wikimedia-cloud) [2020-01-28T10:18:30Z] <arturo> [codfw1dev] created DNS record bastion-codfw1dev-01.codfw1dev.wmcloud.org A 185.15.57.2 (T242976, T229441)

Change 569230 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloud: hiera: puppetmaster: refactor hiera

https://gerrit.wikimedia.org/r/569230

Change 569230 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] cloud: hiera: puppetmaster: refactor hiera (for VM instances)

https://gerrit.wikimedia.org/r/569230

Change 571562 had a related patch set uploaded (by Andrew Bogott; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloud: hiera: puppetmaster: refactor hiera (for VM instances)

https://gerrit.wikimedia.org/r/571562

Change 571562 merged by Andrew Bogott:
[operations/puppet@production] cloud: hiera: puppetmaster: refactor hiera (for VM instances)

https://gerrit.wikimedia.org/r/571562

Change 572243 had a related patch set uploaded (by Andrew Bogott; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] cloud: hiera: puppetmaster: refactor hiera (for VM instances)

https://gerrit.wikimedia.org/r/572243

Change 572243 merged by Andrew Bogott:
[operations/puppet@production] cloud: hiera: puppetmaster: refactor hiera (for VM instances)

https://gerrit.wikimedia.org/r/572243

Change 572268 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Cloud puppet hiera: rename all our 'host' trees to 'hosts'

https://gerrit.wikimedia.org/r/572268

Change 572268 merged by Andrew Bogott:
[operations/puppet@production] Cloud puppet hiera: rename all our 'host' trees to 'hosts'

https://gerrit.wikimedia.org/r/572268

Andrew updated the task description. (Show Details)