Page MenuHomePhabricator
Create Task
Maniphest T104575

Don't rely on wikitech API for production services
Closed, ResolvedPublic
Actions

Description

nfs-project-exports-daemon (presumably among others) hits the wikitech API to learn about instances and their IPs. This is extremely fragile and relies on tech that we've already planned to phase out.

Details

SubjectRepoBranchLines +/-
toollabs: Don't use wikitech API to find labs instances in tools-clush-generatoroperations/puppetproduction+29 -14
Labstore: set openstack::version: mitaka on 1004 and 1005operations/puppetproduction+2 -0
labstore: Don't use wikitech API to find labs instances in nfs-exportdoperations/puppetproduction+41 -23
Follow-up I94eb86ba: Ignore projects where we can't list instancesoperations/puppetproduction+5 -1
graphite: Don't use wikitech API to find labs projects/instancesoperations/puppetproduction+28 -29
graphite: Don't use wikitech API to find labs projects/instancesoperations/puppetproduction+28 -29
Customize query in gerrit

Related Objects
Search...

Event Timeline

Andrew created this task.Jul 2 2015, 2:27 PM
Andrew raised the priority of this task from to Needs Triage.
Andrew updated the task description. (Show Details)
Andrew added projects: Cloud-Services, Cloud-VPS.
Andrew subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 2 2015, 2:27 PM
yuvipanda subscribed.Jul 2 2015, 2:28 PM

+1 - we need a readonly nova api account, I think.

Krenair subscribed.Jul 2 2015, 2:28 PM
• chasemp triaged this task as Medium priority.Nov 30 2015, 4:31 PM
• chasemp subscribed.
• AlexMonk-WMF subscribed.EditedAug 15 2016, 2:18 AM

modules/labstore/files/nfs-exports-daemon: url = ("https://wikitech.wikimedia.org/w/api.php" +

Labs hiera stuff:
modules/wmflib/lib/hiera/mwcache.rb: @httphost = config[:host] || 'https://wikitech.wikimedia.org'
modules/puppetmaster/files/labs.hiera.yaml: :host: https://wikitech.wikimedia.org
modules/puppetmaster/files/labtest.hiera.yaml: :host: https://wikitech.wikimedia.org

Hiera variable status_wiki_host_master - this is more about labs updating wikitech, it's not vital
Hiera variable labs_osm_host - this is to allow silver to connect to ports 5000 and 35357 (keystone) + 9292 (glance) on control, and port 9001 (designate) on services

modules/ldap/manifests/role/config.pp: 'wikicontrollerapiurl' => 'https://wikitech.wikimedia.org/w/api.php', - unused?

modules/graphite/files/archive-instances: 'https://wikitech.wikimedia.org/w/api.php' +
modules/graphite/files/archive-instances: 'https://wikitech.wikimedia.org/w/api.php?action=query&list=novaprojects&format=json'
modules/prometheus/files/usr/local/bin/prometheus-labs-targets: 'https://wikitech.wikimedia.org/w/api.php?' +

yuvipanda added a subtask: T143136: Dump instance info as a static file updated periodically.Aug 16 2016, 6:18 PM
• AlexMonk-WMF mentioned this in T141408: notify_maintainers.py (and, therefore, puppet_alert.py) relies on stale ldap data, needs keystone access.Sep 17 2016, 4:37 AM
gerritbot subscribed.Dec 22 2016, 1:50 AM

Change 328608 had a related patch set uploaded (by Alex Monk):
graphite: Don't use wikitech API to find labs projects/instances

https://gerrit.wikimedia.org/r/328608

gerritbot added a project: Patch-For-Review.Dec 22 2016, 1:50 AM
gerritbot added a comment.Dec 22 2016, 2:09 AM

Change 328609 had a related patch set uploaded (by Alex Monk):
labstore: Don't use wikitech API to find labs instances in nfs-exportd

https://gerrit.wikimedia.org/r/328609

gerritbot added a comment.Dec 24 2016, 12:42 AM

Change 329021 had a related patch set uploaded (by Alex Monk):
toollabs: Don't use wikitech API to find labs instances in tools-clush-generator

https://gerrit.wikimedia.org/r/329021

Andrew closed subtask T104588: Give 'novaobserver' keystone account rights to read everything, everywhere, write or change nothing as Resolved.Jan 17 2017, 8:06 PM
gerritbot added a comment.Jan 19 2017, 6:31 PM

Change 329021 merged by Andrew Bogott:
toollabs: Don't use wikitech API to find labs instances in tools-clush-generator

https://gerrit.wikimedia.org/r/329021

gerritbot added a comment.Jan 20 2017, 2:55 PM

Change 328608 merged by Andrew Bogott:
graphite: Don't use wikitech API to find labs projects/instances

https://gerrit.wikimedia.org/r/328608

gerritbot added a comment.Jan 20 2017, 3:39 PM

Change 333267 had a related patch set uploaded (by Andrew Bogott):
graphite: Don't use wikitech API to find labs projects/instances

https://gerrit.wikimedia.org/r/333267

gerritbot added a comment.Jan 20 2017, 4:28 PM

Change 333267 merged by Andrew Bogott:
graphite: Don't use wikitech API to find labs projects/instances

https://gerrit.wikimedia.org/r/333267

gerritbot added a comment.Jan 20 2017, 7:32 PM

Change 333289 had a related patch set uploaded (by Alex Monk):
Follow-up I94eb86ba: Ignore projects where we can't list instances

https://gerrit.wikimedia.org/r/333289

gerritbot added a comment.Jan 20 2017, 7:37 PM

Change 333289 merged by Andrew Bogott:
Follow-up I94eb86ba: Ignore projects where we can't list instances

https://gerrit.wikimedia.org/r/333289

gerritbot added a comment.Feb 3 2017, 9:41 PM

Change 328609 merged by Andrew Bogott:
labstore: Don't use wikitech API to find labs instances in nfs-exportd

https://gerrit.wikimedia.org/r/328609

gerritbot added a comment.Feb 3 2017, 9:52 PM

Change 335917 had a related patch set uploaded (by Andrew Bogott):
Labstore: set openstack::version: mitaka on 1004 and 1005

https://gerrit.wikimedia.org/r/335917

gerritbot added a comment.Feb 3 2017, 10:00 PM

Change 335917 merged by Andrew Bogott:
Labstore: set openstack::version: mitaka on 1004 and 1005

https://gerrit.wikimedia.org/r/335917

Andrew closed this task as Resolved.Feb 3 2017, 10:21 PM
Andrew claimed this task.

I think that's all of them!

• Phabricator_maintenance removed a subscriber: yuvipanda.Jun 7 2017, 6:49 PM
• Bstorm closed subtask T143136: Dump instance info as a static file updated periodically as Declined.Feb 11 2020, 5:39 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits