Triaging as high as this might bite us and cause issues. We should investigate more and act accordingly
Ι 'll echo Moritz on this one. It does look like adding system users to the admin module adds some complexity and does break the principle of least surprise as well as the KISS principle. I am wondering how much overhead it would be to create system users and groups in the puppetization of the service and provide the access required via that.
Wed, Jan 17
Scheduling this for February 12th 2018, say 10:00 am UTC. I 'll run a few more tests and then send an informational message to engineering@ and wikitech@ and possibly also add a smaller banner to the Home dashboard in grafana.
Tue, Jan 16
Talk on IRC suggests engineering@. It has 202 subscribers so it's probably a better candidate than ops@
The only reason I can think of is people still navigating to grafana-admin and using since it will still DTRT.
Mon, Jan 15
Patchsets above clean up puppetization, drop the ugly distinction of labs vs production from code, moving that into hiera and enable LDAP in production, while disabling the proxy auth. Things still required are
Tue, Jan 9
This should finally be resolved with the above changes
Mon, Jan 8
T180784 has some interesting discussion as well.
Yeah, this has uncovered an unfortunate issue in our decomissioning/reimaging process. See T184444 for more info
Fri, Jan 5
After some experimentation it looks like the main thread is just waiting for the other threads to terminate. This can never happen in normal conditions as they are effectively forever loops. Setting them as daemon threads  allows the mainthread to terminate successfully and the entire python application to terminate as well. Then systemd will takeover and restart it
After some minor changes here and there I did a gdb on the thing and after forcefully closing the TCP connectio to the IRC server we get