Since upgrading to puppet 4 our puppet masters have been logging various deprecation warnings. We should address them and then prevent new instances from being committed.
Here are some ideas and steps to get there
# First, we need a view of currently outstanding deprecation warnings. These can be obtained crudely by parsing `/var/log/syslog` on a puppetmaster, but a more elegant way would be ideal. A Kibana visualization is one way to go. To accomplish that we would need to...
* Ship puppet master syslogs to logstash - There are a few ways to accomplish this each with their own pros/cons so I think it's best suited for a separate task.
* Parse as necessary in logstash - At a minimum extract the standard syslog fields.
* Build a puppet deprecation warnings Kibana visualization .
# Work through outstanding warnings until there are no more deprecation warnings - Since these warnings are logged during each puppet run "no more" means "there hasn't been a deprecation warning logged in N time units". I think ~1 week should be long enough. It needs to be balanced between long enough to catch outliers (disabled agents, etc.) but short enough to stay on top of new offenders committed while this effort is under way.
# Then, once we're not seeing deprecation warnings logged, enable `fail_on_deprecation_notices` as discussed in T154915