Page MenuHomePhabricator
Create Task
Maniphest T185239

Puppet hosts with signed certificate present on agent but not master
Closed, ResolvedPublic
Actions

Description

After deploying the Apache CRL check setting to address T184444 several hosts have been found in an unusual state where:

  • a signed certificate is present on the agent/server and...
  • the certificate has not been revoked but...
  • the signed certificate is not present on the puppet master

Normally this could be addressed simply by generating and signing new certificates, but this is complicated by the fact that systems are exposing (via base::expose_puppet_certs) their puppet certificates for use in other applications.

Below are systems exposing their puppet cert/key for use in other applications. Let's review each for impact and potential issues before proceeding with generating/signing new puppet certs.

(check off after new puppet cert has been generated and signed)

  • cp1052.eqiad.wmnet - decommissioned T208584
  • es2014.codfw.wmnet - decommissioned T262889
  • ganeti2001.codfw.wmnet
  • ms-be2021.codfw.wmnet - decommissioned T272837
  • mw2105.codfw.wmnet - decommissioned (cant find task)
  • mw2121.codfw.wmnet - decommissioned T189111
  • mw2131.codfw.wmnet - decommissioned T189111
  • mw2132.codfw.wmnet - decommissioned T189111
  • mw2144.codfw.wmnet - decommissioned T261524
  • mw2191.codfw.wmnet - decommissioned T261524
  • mw2193.codfw.wmnet - decommissioned T261524
  • mw2206.codfw.wmnet - decommissioned T261524
  • mw2229.codfw.wmnet - decommissioned T277119
  • mw2231.codfw.wmnet - decommissioned T277119
  • mw2240.codfw.wmnet - decommissioned T277119
  • rdb2002.codfw.wmnet - decommissioned T209425
  • wtp2005.codfw.wmnet
  • wtp2020.codfw.wmnet

Related Objects
Search...

Event Timeline

herron triaged this task as Medium priority.Jan 18 2018, 6:41 PM
herron created this task.
herron updated the task description. (Show Details)Jan 18 2018, 6:44 PM
• ema added a project: Traffic.Jan 18 2018, 7:07 PM
• ema moved this task from Backlog to General on the Traffic board.
herron updated the task description. (Show Details)Jan 22 2018, 2:06 PM
herron mentioned this in T184444: Puppet hosts with their cert revoked can still run puppet.Jan 22 2018, 2:16 PM
akosiaris added a subscriber: akosiaris.Jan 22 2018, 5:52 PM

ganeti2001, wtp2005, wtp2020 only use that cert for rsyslog and not for any service so we can refresh the puppet certs for them whenever we want

herron updated the task description. (Show Details)Jan 22 2018, 8:29 PM

Thanks @akosiaris! ganeti2001, wtp2005, wtp2020 puppet certs have been refreshed.

herron added subscribers: • ema, elukey.Feb 2 2018, 6:02 PM
herron added a project: User-herron.Jun 20 2018, 3:30 PM
• Phabricator_maintenance moved this task from Backlog to Acknowledged on the SRE board.Jan 26 2019, 9:43 PM
Volans mentioned this in T250483: Puppet certificate discrepancies.Apr 17 2020, 1:10 PM
Aklapper removed herron as the assignee of this task.Jun 19 2020, 4:19 PM

This task has been assigned to the same task owner for more than two years. Resetting task assignee due to inactivity, to decrease task cookie-licking and to get a slightly more realistic overview of plans. Please feel free to assign this task to yourself again if you still realistically work or plan to work on this task - it would be welcome!

For tips how to manage individual work in Phabricator (noisy notifications, lists of task, etc.), see https://phabricator.wikimedia.org/T228575#6237124 for available options.
(For the records, two emails were sent to assignee addresses before resetting assignees. See T228575 for more info and for potential feedback. Thanks!)

BBlack updated the task description. (Show Details)Sep 29 2020, 8:49 PM
BBlack removed a project: Traffic.
BBlack added a subscriber: BBlack.

lvs100[789] don't exist anymore, removing Traffic from this.

Aklapper added a project: Infrastructure-Foundations.Jun 21 2021, 9:00 PM
jbond closed this task as Resolved.Aug 4 2022, 11:34 AM
jbond claimed this task.
jbond updated the task description. (Show Details)
jbond added a subscriber: jbond.

Closing all servers listed have been decomissioned

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL