Page MenuHomePhabricator

Horizon dashboard for managing http proxies for labs instances
Closed, ResolvedPublic

Description

The api for this used to be called 'invisible unicorn' -- not sure if it's renamed yet. It's a proper API registered in the keystone catalog, so hooking things up to horizon should be straightforward.

Event Timeline

Having thought about this for 30 seconds, I'm remembering that the SpecialNovaProxy interface actually manages /two/ things: the actual proxy (via invisible unicorn) and the dns record for the proxy.

The new GUI should handle the dns side by making designate calls rather than ldap manipulation. That will be much easier to implement, but will be a bit confusing in the transitional period when public dns isn't actually live on designate/horizon. When you add records to designate they probably won't show up for ping or dig since right now the live servers are labs-ns0/ns1, but they should show up if you dig @labs-ns2/ns3.

Change 276893 had a related patch set uploaded (by Alex Monk):
horizon: Add dynamicproxy IPs to config

https://gerrit.wikimedia.org/r/276893

Status update: The basic dynamicproxy - horizon integration is working, the challenge now is dealing with the DNS record and designate's rules about ownership. Then we should be able to add it to labtesthorizon (probably just going to be a matter of adding a bunch of files via puppet), then add docs and user instructions on the forms, policy checks, etc. and hopefully put it in 'production labs'.

Change 276893 abandoned by Alex Monk:
horizon: Add dynamicproxy IPs to config

Reason:
in favour of using the keystone endpoint thing

https://gerrit.wikimedia.org/r/276893

Change 278538 had a related patch set uploaded (by Alex Monk):
Add wmflabsdotorg credentials to horizon config

https://gerrit.wikimedia.org/r/278538

This has been on labtesthorizon for a little while now. The changes aren't puppetised yet but that's only actively an issue for the config change above (have to disable puppet and re-make the config change while testing it) - the rest is just new files (which puppet is fine with).
I'd like @Andrew to check the policy rules stuff, and we should probably add a helpful description to the index (?) and create proxy form, then the new files can be puppetised and the panel shown to users on the real horizon when we're ready to do the migration.
After that:

  • See if we can return status/text upon deletion failure
  • Batch deletion? There are checkboxes that do nothing at the moment
  • Pick a single Designate API version and stick with it. Currently we use both v1 (normal case) and v2 (proxies on sub-domains of wmflabs.org itself, not sub-sub-domains). v2 is obviously the way forward but may or may not be possible (might be easy, haven't looked into it) to use with designate-dashboard

Regarding policies... I think we can just re-use the create/delete record policies from designate. If the proxy policies were ever less restrictive than the designate policies then we'd run afoul of them anyway. So that's just:

policy_rules = (("dns", "create_record"),)
and
policy_rules = (("dns", "delete_record"),)

Other random security/validation thoughts:

  • Thanks to our shared-credential hack, I'm worried that improper input to delete() could result in removal of wmflabs.org records from other projects. There are a few ways we could avoid this... something like https://phabricator.wikimedia.org/T129800 for deletion as well as creation would give us a good way to validate ownership; there are probably other ways we could have that deletion code check for ownership.
  • I'd like to validate the domain name being created -- I don't know offhand what the official permitted character set is for domains, but we should at least forbid dots.

I'd also like to see this code in a gerrit patch ASAP -- as a backup mechanism if nothing else.

Change 278871 had a related patch set uploaded (by Alex Monk):
[WIP] openstack: Add proxy panel files

https://gerrit.wikimedia.org/r/278871

I'd also like to see this code in a gerrit patch ASAP -- as a backup mechanism if nothing else.

I did have a copy on labtestweb2001, older files in real labs and an up to date copy on my laptop, but the WIP puppet patch is up now. Haven't tested it

Change 278538 merged by Andrew Bogott:
Add wmflabsdotorg credentials to horizon config

https://gerrit.wikimedia.org/r/278538

Change 278871 merged by Andrew Bogott:
openstack: Add proxy panel files

https://gerrit.wikimedia.org/r/278871

this is now merged and running on horizon.wikimedia.org and looks pretty good!

Shall we still split views.py up? Or just close this as done?

The dashboard works and is installed on production (although temporarily disabled during a dns migration).

Alex, you're welcome to tidy things up if you feel like it, and bill us for the time. Meanwhile I'm going to close this bug; we can open more specific tickets as issues arise.