Move XHGui from tungsten to webperf-002
Open, NormalPublic

Description

(This task was originally only about setting up XHGui in Beta Cluster).

  • Create "webperf-002" production instances: T194390.
  • Create Beta Cluster instance: deployment-webperf12.
  • Create Puppet role class for the instance: role::webperf::profiling_tools, and apply to the above.
  • Improve xhgui puppet classes and profile to account for multi-dc.
  • Add xhgui profile to role::webperf::profiling_tools.
  • Set xhgui destination in wmf-config/profiler-labs.php
Krinkle created this task.Nov 17 2017, 12:17 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 17 2017, 12:17 AM
Krinkle updated the task description. (Show Details)Nov 17 2017, 12:18 AM
Krinkle triaged this task as Low priority.Nov 20 2017, 9:16 PM
Krinkle updated the task description. (Show Details)Jan 31 2018, 2:59 AM
Krinkle updated the task description. (Show Details)
Krinkle renamed this task from Set up XHGui for Beta Cluster to Move XHGui from tungsten to webperf-002.Jul 17 2018, 1:36 AM
Krinkle raised the priority of this task from Low to Normal.
Krinkle assigned this task to Imarlier.
Krinkle updated the task description. (Show Details)
Stashbot added a subscriber: Stashbot.

Mentioned in SAL (#wikimedia-releng) [2018-07-17T01:36:42Z] <Krinkle> Applying role::webperf::profiling_tools class to webperf12 in Beta Cluster - T195312, T180761.

Applying the role, while seemingly fine in prod, seems to have failed in Beta Cluster for reasons not clear to me.

1puppet-agent-cronjob: Sleeping 0 for random splay
2
3puppet-agent[20904]: Using configured environment 'production'
4puppet-agent[20904]: Retrieving pluginfacts
5puppet-agent[20904]: Retrieving plugin
6puppet-agent[20904]: Loading facts
7puppet-agent[20904]: Caching catalog for deployment-webperf12.deployment-prep.eqiad.wmflabs
8puppet-agent[20904]: (/Stage[main]/Base::Environment/Tidy[/var/tmp/core]) Tidying 0 files
9puppet-agent[20904]: Applying configuration version '1531791435'
10puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/modprobe.d/nf_conntrack.conf]/ensure) defined content as '{md5}401f0bc2c4f44a223e309be434d2c3b4'
11puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/modules-load.d/conntrack.conf]/ensure) defined content as '{md5}65a23fdacaecc29912d86ebbd1256077'
12
13systemd[1]: Reloading.
14systemd[1]: Reloading.
15systemd[1]: Reloading.
16systemd[1]: Reloading.
17systemd[1]: Starting ferm firewall configuration...
18systemd-udevd[201]: Network interface NamePolicy= disabled on kernel command line, ignoring.
19kernel: [1201819.399283] nf_conntrack version 0.5.0 (32768 buckets, 262144 max)
20ferm[21430]: Starting Firewall: ferm.
21systemd[1]: Started ferm firewall configuration.
22systemd[1]: Reloading.
23
24puppet-agent[20904]: (/Stage[main]/Ferm/Package[ferm]/ensure) created
25puppet-agent[20904]: (/Stage[main]/Ferm/Package[libnet-dns-perl]/ensure) created
26puppet-agent[20904]: (/Stage[main]/Ferm/Package[conntrack]/ensure) created
27puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content)
28puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) --- /etc/ferm/ferm.conf#0112017-06-06 10:40:08.000000000 +0000
29puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) +++ /tmp/puppet-file20180717-20904-16wsuuk#0112018-07-17 01:37:35.111144952 +0000
30puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) @@ -1,52 +1,3 @@
31puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -# -*- shell-script -*-
32puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -#
33puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -# Configuration file for ferm(1).
34puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -#
35puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) +@include 'functions.conf';
36puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content)
37puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -table filter {
38puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - chain INPUT {
39puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - policy DROP;
40puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -
41puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - # connection tracking
42puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - mod state state INVALID DROP;
43puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - mod state state (ESTABLISHED RELATED) ACCEPT;
44puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -
45puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - # allow local packet
46puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - interface lo ACCEPT;
47puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -
48puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - # respond to ping
49puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - proto icmp ACCEPT;
50puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -
51puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - # allow IPsec
52puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - proto udp dport 500 ACCEPT;
53puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - proto (esp ah) ACCEPT;
54puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -
55puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - # allow SSH connections
56puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - proto tcp dport ssh ACCEPT;
57puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - }
58puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - chain OUTPUT {
59puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - policy ACCEPT;
60puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -
61puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - # connection tracking
62puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - #mod state state INVALID DROP;
63puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - mod state state (ESTABLISHED RELATED) ACCEPT;
64puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - }
65puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - chain FORWARD {
66puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - policy DROP;
67puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -
68puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - # connection tracking
69puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - mod state state INVALID DROP;
70puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - mod state state (ESTABLISHED RELATED) ACCEPT;
71puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) - }
72puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -}
73puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -
74puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -# IPv6:
75puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -#domain ip6 {
76puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -# table filter {
77puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -# chain INPUT {
78puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -# policy ACCEPT;
79puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -# # ...
80puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -# }
81puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -# # ...
82puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -# }
83puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) -#}
84puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) +@include 'conf.d/';
85puppet-agent[20904]: Computing checksum on file /etc/ferm/ferm.conf
86puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]) Filebucketed /etc/ferm/ferm.conf to puppet with sum 91410f27613e600a8892d2a7076d1bcf
87puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/content) content changed '{md5}91410f27613e600a8892d2a7076d1bcf' to '{md5}4bea2934a124683725db912836697b1a'
88puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/group) group changed 'adm' to 'root'
89puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]/mode) mode changed '0644' to '0400'
90puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]) Scheduling refresh of Service[ferm]
91puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]) Scheduling refresh of Service[ferm]
92puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/ferm.conf]) Scheduling refresh of Service[ferm]
93puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/functions.conf]/ensure) defined content as '{md5}cfb158d05f0678fd3f282fa73f63f87d'
94puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/functions.conf]) Scheduling refresh of Service[ferm]
95puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/conf.d]/ensure) created
96puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/ferm/conf.d]) Scheduling refresh of Service[ferm]
97puppet-agent[20904]: (/Stage[main]/Role::Prometheus::Node_exporter/Ferm::Service[prometheus-node-exporter]/File[/etc/ferm/conf.d/10_prometheus-node-exporter]/ensure) defined content as '{md5}0d82d468a63b2bad2a2dc8ae395aa408'
98puppet-agent[20904]: (/Stage[main]/Role::Prometheus::Node_exporter/Ferm::Service[prometheus-node-exporter]/File[/etc/ferm/conf.d/10_prometheus-node-exporter]) Scheduling refresh of Service[ferm]
99puppet-agent[20904]: (/Stage[main]/Profile::Openstack::Main::Cumin::Target/Ferm::Service[ssh-from-cumin-project-masters]/File[/etc/ferm/conf.d/10_ssh-from-cumin-project-masters]/ensure) defined content as '{md5}26cce1eb911fc807e27cf02cce9b40a0'
100puppet-agent[20904]: (/Stage[main]/Profile::Openstack::Main::Cumin::Target/Ferm::Service[ssh-from-cumin-project-masters]/File[/etc/ferm/conf.d/10_ssh-from-cumin-project-masters]) Scheduling refresh of Service[ferm]
101puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content)
102puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) --- /etc/default/ferm#0112018-07-17 01:37:30.287171005 +0000
103puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) +++ /tmp/puppet-file20180717-20904-z3idqf#0112018-07-17 01:37:35.527142706 +0000
104puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) @@ -4,10 +4,11 @@
105puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) FAST=yes
106puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content)
107puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) # cache the output of ferm --lines in /var/cache/ferm?
108puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) -CACHE=yes
109puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) +CACHE=no
110puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content)
111puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) -# additional paramaters for ferm (like --def '=bar')
112puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) +# additional paramaters for ferm (like --def '$foo=bar')
113puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) OPTIONS=
114puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content)
115puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) -# Enable the ferm init script? (i.e. run on bootup)
116puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) -ENABLED="yes"
117puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) +# Enable ferm on bootup?
118puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) +ENABLED=yes
119puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) +
120puppet-agent[20904]: Computing checksum on file /etc/default/ferm
121puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]) Filebucketed /etc/default/ferm to puppet with sum a4daba7939f6be9a87f26f1a89324806
122puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/content) content changed '{md5}a4daba7939f6be9a87f26f1a89324806' to '{md5}3e9b11c20066c1658ab353e597ea8e5e'
123puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]/mode) mode changed '0644' to '0400'
124puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]) Scheduling refresh of Service[ferm]
125puppet-agent[20904]: (/Stage[main]/Ferm/File[/etc/default/ferm]) Scheduling refresh of Service[ferm]
126puppet-agent[20904]: (/Stage[main]/Base::Firewall/File[/usr/lib/nagios/plugins/check_conntrack]/ensure) defined content as '{md5}a5a705cc6b0bc5e9074c467d69ae3eb3'
127puppet-agent[20904]: (/Stage[main]/Base::Firewall/File[/usr/lib/nagios/plugins/check_ferm]/ensure) defined content as '{md5}f584c405e02f63bc7321239204993c75'
128
129systemd[1]: Reloading.
130nscd: 5328 monitoring file `/etc/passwd` (7)
131nscd: 5328 monitoring directory `/etc` (2)
132nscd: 5328 monitoring file `/etc/group` (8)
133nscd: 5328 monitoring directory `/etc` (2)
134nscd: 5328 monitored file `/etc/group` was written to
135nscd: 5328 monitored file `/etc/group` was moved into place, adding watch
136nscd: 5328 monitoring file `/etc/passwd` (7)
137nscd: 5328 monitoring directory `/etc` (2)
138nscd: 5328 monitoring file `/etc/group` (9)
139nscd: 5328 monitoring directory `/etc` (2)
140nscd: 5328 monitoring file `/etc/group` (9)
141nscd: 5328 monitoring directory `/etc` (2)
142nscd: 5328 monitoring file `/etc/group` (9)
143nscd: 5328 monitoring directory `/etc` (2)
144nscd: 5328 monitored file `/etc/passwd` was written to
145nscd: 5328 monitored file `/etc/passwd` was moved into place, adding watch
146nscd: 5328 monitored file `/etc/group` was written to
147nscd: 5328 monitoring file `/etc/passwd` (10)
148nscd: 5328 monitoring directory `/etc` (2)
149nscd: 5328 monitoring file `/etc/group` (9)
150nscd: 5328 monitoring directory `/etc` (2)
151nscd: 5328 monitoring file `/etc/passwd` (10)
152nscd: 5328 monitoring directory `/etc` (2)
153nscd: 5328 monitoring file `/etc/group` (9)
154nscd: 5328 monitoring directory `/etc` (2)
155nscd: 5328 monitored file `/etc/passwd` was written to
156nscd: 5328 monitoring file `/etc/passwd` (10)
157nscd: 5328 monitoring directory `/etc` (2)
158nscd: 5328 monitoring file `/etc/group` (9)
159nscd: 5328 monitoring directory `/etc` (2)
160nscd: 5328 monitoring file `/etc/passwd` (10)
161nscd: 5328 monitoring directory `/etc` (2)
162nscd: 5328 monitoring file `/etc/group` (9)
163nscd: 5328 monitoring directory `/etc` (2)
164systemd[1]: Reloading.
165systemd[1]: Reloading.
166systemd[1]: Reloading.
167systemd[1]: Started An object/document-oriented database.
168systemd[1]: Reloading.
169
170puppet-agent[20904]: (/Stage[main]/Mongodb/Package[mongodb]/ensure) created
171puppet-agent[20904]: (/Stage[main]/Mongodb/File[/srv/mongod]/ensure) created
172puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content)
173puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) --- /etc/mongodb.conf#0112017-08-12 16:10:53.000000000 +0000
174puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) +++ /tmp/puppet-file20180717-20904-1ma88h0#0112018-07-17 01:37:46.611082860 +0000
175puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) @@ -1,100 +1,9 @@
176puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# mongodb.conf
177puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) +# Configuration file for MongoDB
178puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) +# <http://docs.mongodb.org/manual/reference/configuration-options/>
179puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) +# This file is managed by Puppet.
180puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) +dbpath = /srv/mongod
181puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) +fork = false
182puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) +logappend = true
183puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) +logpath = /var/log/mongodb/mongodb.log
184puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content)
185puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Where to store the data.
186puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -dbpath=/var/lib/mongodb
187puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
188puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#where to log
189puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -logpath=/var/log/mongodb/mongodb.log
190puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
191puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -logappend=true
192puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
193puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -bind_ip = 127.0.0.1
194puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#port = 27017
195puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
196puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Enable journaling, http://www.mongodb.org/display/DOCS/Journaling
197puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -journal=true
198puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
199puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Enables periodic logging of CPU utilization and I/O wait
200puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#cpu = true
201puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
202puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Turn on/off security. Off is currently the default
203puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#noauth = true
204puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#auth = true
205puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
206puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Verbose logging output.
207puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#verbose = true
208puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
209puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Inspect all client data for validity on receipt (useful for
210puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# developing drivers)
211puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#objcheck = true
212puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
213puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Enable db quota management
214puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#quota = true
215puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
216puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Set oplogging level where n is
217puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# 0=off (default)
218puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# 1=W
219puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# 2=R
220puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# 3=both
221puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# 7=W+some reads
222puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#oplog = 0
223puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
224puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Diagnostic/debugging option
225puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#nocursors = true
226puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
227puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Ignore query hints
228puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#nohints = true
229puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
230puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Disable the HTTP interface (Defaults to localhost:27018).
231puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#nohttpinterface = true
232puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
233puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Turns off server-side scripting. This will result in greatly limited
234puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# functionality
235puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#noscripting = true
236puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
237puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Turns off table scans. Any query that would do a table scan fails.
238puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#notablescan = true
239puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
240puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Disable data file preallocation.
241puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#noprealloc = true
242puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
243puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Specify .ns file size for new databases.
244puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# nssize = <size>
245puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
246puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Accout token for Mongo monitoring server.
247puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#mms-token = <token>
248puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
249puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Server name for Mongo monitoring server.
250puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#mms-name = <server-name>
251puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
252puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Ping interval for Mongo monitoring server.
253puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#mms-interval = <seconds>
254puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
255puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Replication Options
256puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
257puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# in replicated mongo databases, specify here whether this is a slave or master
258puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#slave = true
259puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#source = master.example.com
260systemd[1]: Stopping An object/document-oriented database...
261puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Slave only: specify a single database to replicate
262puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#only = master.example.com
263puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# or
264puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#master = true
265puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#source = slave.example.com
266puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
267puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Address of a server to pair with.
268puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#pairwith = <server:port>
269puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Address of arbiter server.
270puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#arbiter = <server:port>
271puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Automatically resync if slave data is stale
272puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#autoresync
273puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Custom size for replication operation log.
274puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#oplogSize = <MB>
275puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Size limit for in-memory storage of op ids.
276puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#opIdMem = <bytes>
277puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -
278puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# SSL options
279puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# Enable SSL on normal ports
280puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#sslOnNormalPorts = true
281puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -# SSL Key file and password
282puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#sslPEMKeyFile = /etc/ssl/mongodb.pem
283puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) -#sslPEMKeyPassword = pass
284puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) +# vim: set ft=eruby.conf:
285puppet-agent[20904]: Computing checksum on file /etc/mongodb.conf
286puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]) Filebucketed /etc/mongodb.conf to puppet with sum f3827b0188f8ce594abb3ebee1808027
287puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]/content) content changed '{md5}f3827b0188f8ce594abb3ebee1808027' to '{md5}452f76d03d4c428670d56da7db10cf0b'
288puppet-agent[20904]: (/Stage[main]/Mongodb/File[/etc/mongodb.conf]) Scheduling refresh of Service[mongodb]
289systemd[1]: Stopped An object/document-oriented database.
290puppet-agent[20904]: (/Stage[main]/Mongodb/Service[mongodb]) Triggered 'refresh' from 1 events
291
292systemd[1]: Started An object/document-oriented database.
293systemd[1]: Reloading.
294systemd[1]: Reloading.
295systemd[1]: Started Clean PHP session files every 30 mins.
296
297php7.0-common: php_invoke calendar: already enabled for PHP 7.0 apache2 sapi
298php7.0-common: php_invoke calendar: already enabled for PHP 7.0 cli sapi
299php7.0-common: php_invoke ctype: already enabled for PHP 7.0 apache2 sapi
300php7.0-common: php_invoke ctype: already enabled for PHP 7.0 cli sapi
301php7.0-common: php_invoke exif: already enabled for PHP 7.0 apache2 sapi
302php7.0-common: php_invoke exif: already enabled for PHP 7.0 cli sapi
303php7.0-common: php_invoke fileinfo: already enabled for PHP 7.0 apache2 sapi
304php7.0-common: php_invoke fileinfo: already enabled for PHP 7.0 cli sapi
305php7.0-common: php_invoke ftp: already enabled for PHP 7.0 apache2 sapi
306php7.0-common: php_invoke ftp: already enabled for PHP 7.0 cli sapi
307php7.0-common: php_invoke gettext: already enabled for PHP 7.0 apache2 sapi
308php7.0-common: php_invoke gettext: already enabled for PHP 7.0 cli sapi
309php7.0-common: php_invoke iconv: already enabled for PHP 7.0 apache2 sapi
310php7.0-common: php_invoke iconv: already enabled for PHP 7.0 cli sapi
311php7.0-common: php_invoke pdo: already enabled for PHP 7.0 apache2 sapi
312php7.0-common: php_invoke pdo: already enabled for PHP 7.0 cli sapi
313php7.0-common: php_invoke phar: already enabled for PHP 7.0 apache2 sapi
314php7.0-common: php_invoke phar: already enabled for PHP 7.0 cli sapi
315php7.0-common: php_invoke posix: already enabled for PHP 7.0 apache2 sapi
316php7.0-common: php_invoke posix: already enabled for PHP 7.0 cli sapi
317php7.0-common: php_invoke shmop: already enabled for PHP 7.0 apache2 sapi
318php7.0-common: php_invoke shmop: already enabled for PHP 7.0 cli sapi
319php7.0-common: php_invoke sockets: already enabled for PHP 7.0 apache2 sapi
320php7.0-common: php_invoke sockets: already enabled for PHP 7.0 cli sapi
321php7.0-common: php_invoke sysvmsg: already enabled for PHP 7.0 apache2 sapi
322php7.0-common: php_invoke sysvmsg: already enabled for PHP 7.0 cli sapi
323php7.0-common: php_invoke sysvsem: already enabled for PHP 7.0 apache2 sapi
324php7.0-common: php_invoke sysvsem: already enabled for PHP 7.0 cli sapi
325php7.0-common: php_invoke sysvshm: already enabled for PHP 7.0 apache2 sapi
326php7.0-common: php_invoke sysvshm: already enabled for PHP 7.0 cli sapi
327php7.0-common: php_invoke tokenizer: already enabled for PHP 7.0 apache2 sapi
328php7.0-common: php_invoke tokenizer: already enabled for PHP 7.0 cli sapi
329php7.0-readline: php_invoke readline: already enabled for PHP 7.0 apache2 sapi
330php7.0-readline: php_invoke readline: already enabled for PHP 7.0 cli sapi
331php7.0-opcache: php_invoke opcache: already enabled for PHP 7.0 apache2 sapi
332php7.0-opcache: php_invoke opcache: already enabled for PHP 7.0 cli sapi
333php7.0-json: php_invoke json: already enabled for PHP 7.0 apache2 sapi
334php7.0-json: php_invoke json: already enabled for PHP 7.0 cli sapi
335php7.0-cli: php_invoke sysvsem: already enabled for PHP 7.0 cli sapi
336php7.0-cli: php_invoke calendar: already enabled for PHP 7.0 cli sapi
337php7.0-cli: php_invoke sysvshm: already enabled for PHP 7.0 cli sapi
338php7.0-cli: php_invoke tokenizer: already enabled for PHP 7.0 cli sapi
339php7.0-cli: php_invoke fileinfo: already enabled for PHP 7.0 cli sapi
340php7.0-cli: php_invoke iconv: already enabled for PHP 7.0 cli sapi
341php7.0-cli: php_invoke opcache: already enabled for PHP 7.0 cli sapi
342php7.0-cli: php_invoke readline: already enabled for PHP 7.0 cli sapi
343php7.0-cli: php_invoke json: already enabled for PHP 7.0 cli sapi
344php7.0-cli: php_invoke ctype: already enabled for PHP 7.0 cli sapi
345php7.0-cli: php_invoke exif: already enabled for PHP 7.0 cli sapi
346php7.0-cli: php_invoke shmop: already enabled for PHP 7.0 cli sapi
347php7.0-cli: php_invoke sockets: already enabled for PHP 7.0 cli sapi
348php7.0-cli: php_invoke ftp: already enabled for PHP 7.0 cli sapi
349php7.0-cli: php_invoke gettext: already enabled for PHP 7.0 cli sapi
350php7.0-cli: php_invoke phar: already enabled for PHP 7.0 cli sapi
351php7.0-cli: php_invoke sysvmsg: already enabled for PHP 7.0 cli sapi
352php7.0-cli: php_invoke pdo: already enabled for PHP 7.0 cli sapi
353php7.0-cli: php_invoke posix: already enabled for PHP 7.0 cli sapi
354libapache2-mod-php7.0: php_invoke sysvsem: already enabled for PHP 7.0 apache2 sapi
355libapache2-mod-php7.0: php_invoke calendar: already enabled for PHP 7.0 apache2 sapi
356libapache2-mod-php7.0: php_invoke sysvshm: already enabled for PHP 7.0 apache2 sapi
357libapache2-mod-php7.0: php_invoke tokenizer: already enabled for PHP 7.0 apache2 sapi
358libapache2-mod-php7.0: php_invoke fileinfo: already enabled for PHP 7.0 apache2 sapi
359libapache2-mod-php7.0: php_invoke iconv: already enabled for PHP 7.0 apache2 sapi
360libapache2-mod-php7.0: php_invoke opcache: already enabled for PHP 7.0 apache2 sapi
361libapache2-mod-php7.0: php_invoke readline: already enabled for PHP 7.0 apache2 sapi
362libapache2-mod-php7.0: php_invoke json: already enabled for PHP 7.0 apache2 sapi
363libapache2-mod-php7.0: php_invoke ctype: already enabled for PHP 7.0 apache2 sapi
364libapache2-mod-php7.0: php_invoke exif: already enabled for PHP 7.0 apache2 sapi
365libapache2-mod-php7.0: php_invoke shmop: already enabled for PHP 7.0 apache2 sapi
366libapache2-mod-php7.0: php_invoke sockets: already enabled for PHP 7.0 apache2 sapi
367libapache2-mod-php7.0: php_invoke ftp: already enabled for PHP 7.0 apache2 sapi
368libapache2-mod-php7.0: php_invoke gettext: already enabled for PHP 7.0 apache2 sapi
369libapache2-mod-php7.0: php_invoke phar: already enabled for PHP 7.0 apache2 sapi
370libapache2-mod-php7.0: php_invoke sysvmsg: already enabled for PHP 7.0 apache2 sapi
371libapache2-mod-php7.0: php_invoke pdo: already enabled for PHP 7.0 apache2 sapi
372libapache2-mod-php7.0: php_invoke posix: already enabled for PHP 7.0 apache2 sapi
373
374puppet-agent[20904]: (/Stage[main]/Packages::Libapache2_mod_php7.0/Package[libapache2-mod-php7.0]/ensure) created
375CRON[28359]: (prometheus) CMD (/usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom)
376
377systemd[1]: Reloading.
378systemd[1]: Reloading.
379systemd[1]: Reloading.
380systemd[1]: Reloading.
381systemd[1]: Reloading.
382systemd[1]: Starting The Apache HTTP Server...
383systemd[1]: Started The Apache HTTP Server.
384systemd[1]: Reloading.
385systemd[1]: Reloading.
386systemd[1]: Reloading.
387systemd[1]: Reloading.
388
389puppet-agent[20904]: (/Stage[main]/Packages::Apache2/Package[apache2]/ensure) created
390puppet-agent[20904]: (/Stage[main]/Packages::Links/Package[links]/ensure) created
391puppet-agent[20904]: (/Stage[main]/Httpd/File[/etc/apache2/conf-enabled/charset.conf]/ensure) removed
392puppet-agent[20904]: (/Stage[main]/Httpd/File[/etc/apache2/conf-enabled/localized-error-pages.conf]/ensure) removed
393puppet-agent[20904]: (/Stage[main]/Httpd/File[/etc/apache2/conf-enabled/other-vhosts-access-log.conf]/ensure) removed
394puppet-agent[20904]: (/Stage[main]/Httpd/File[/etc/apache2/conf-enabled/security.conf]/ensure) removed
395puppet-agent[20904]: (/Stage[main]/Httpd/File[/etc/apache2/conf-enabled/serve-cgi-bin.conf]/ensure) removed
396puppet-agent[20904]: (/etc/apache2/conf-enabled) Scheduling refresh of Service[apache2]
397puppet-agent[20904]: (/Stage[main]/Httpd/File[/etc/apache2/env-available]/ensure) created
398puppet-agent[20904]: (/Stage[main]/Httpd/File[/etc/apache2/env-enabled]/ensure) created
399puppet-agent[20904]: (/Stage[main]/Httpd/File[/etc/apache2/env-enabled]) Scheduling refresh of Service[apache2]
400puppet-agent[20904]: (/Stage[main]/Httpd/File[/etc/apache2/sites-enabled/000-default.conf]/ensure) removed
401puppet-agent[20904]: (/etc/apache2/sites-enabled) Scheduling refresh of Service[apache2]
402puppet-agent[20904]: (/Stage[main]/Httpd/File_line[load_env_enabled]/ensure) created
403puppet-agent[20904]: (Augeas[Apache2 logs](provider=augeas))
404puppet-agent[20904]: (Augeas[Apache2 logs](provider=augeas)) --- /etc/logrotate.d/apache2#0112018-06-02 07:48:33.000000000 +0000
405puppet-agent[20904]: (Augeas[Apache2 logs](provider=augeas)) +++ /etc/logrotate.d/apache2.augnew#0112018-07-17 01:38:07.298971209 +0000
406puppet-agent[20904]: (Augeas[Apache2 logs](provider=augeas)) @@ -1,7 +1,7 @@
407puppet-agent[20904]: (Augeas[Apache2 logs](provider=augeas)) /var/log/apache2/*.log {
408puppet-agent[20904]: (Augeas[Apache2 logs](provider=augeas)) #011daily
409puppet-agent[20904]: (Augeas[Apache2 logs](provider=augeas)) #011missingok
410puppet-agent[20904]: (Augeas[Apache2 logs](provider=augeas)) -#011rotate 14
411puppet-agent[20904]: (Augeas[Apache2 logs](provider=augeas)) +#011rotate 30
412puppet-agent[20904]: (Augeas[Apache2 logs](provider=augeas)) #011compress
413puppet-agent[20904]: (Augeas[Apache2 logs](provider=augeas)) #011delaycompress
414puppet-agent[20904]: (Augeas[Apache2 logs](provider=augeas)) #011notifempty
415puppet-agent[20904]: (/Stage[main]/Httpd/Augeas[Apache2 logs]/returns) executed successfully
416puppet-agent[20904]: Computing checksum on file /etc/apache2/mods-available/status.conf
417puppet-agent[20904]: (/Stage[main]/Httpd/File[/etc/apache2/mods-available/status.conf]) Filebucketed /etc/apache2/mods-available/status.conf to puppet with sum 5d3689f109baf304af4f145249ca6326
418puppet-agent[20904]: (/Stage[main]/Httpd/File[/etc/apache2/mods-available/status.conf]/ensure) removed
419puppet-agent[20904]: (/Stage[main]/Httpd/File[/etc/apache2/mods-enabled/status.conf]/ensure) removed
420puppet-agent[20904]: (/Stage[main]/Httpd/File[/usr/local/bin/apache-status]/ensure) defined content as '{md5}fb80019f03331dde9239e9a6a0af0d46'
421puppet-agent[20904]: (/Stage[main]/Base::Firewall/Ferm::Conf[defs]/File[/etc/ferm/conf.d/00_defs]/ensure) defined content as '{md5}b3f157a6aaa42c2299e9f567af937174'
422puppet-agent[20904]: (/Stage[main]/Base::Firewall/Ferm::Conf[defs]/File[/etc/ferm/conf.d/00_defs]) Scheduling refresh of Service[ferm]
423puppet-agent[20904]: (/Stage[main]/Base::Firewall/Ferm::Conf[main]/File[/etc/ferm/conf.d/00_main]/ensure) defined content as '{md5}31835a68d4305c91a1acf38e37eb827d'
424puppet-agent[20904]: (/Stage[main]/Base::Firewall/Ferm::Conf[main]/File[/etc/ferm/conf.d/00_main]) Scheduling refresh of Service[ferm]
425puppet-agent[20904]: (/Stage[main]/Base::Firewall/Ferm::Rule[bastion-ssh]/File[/etc/ferm/conf.d/10_bastion-ssh]/ensure) defined content as '{md5}96b0e1a82b89be89ff332b32f3e768fc'
426puppet-agent[20904]: (/Stage[main]/Base::Firewall/Ferm::Rule[bastion-ssh]/File[/etc/ferm/conf.d/10_bastion-ssh]) Scheduling refresh of Service[ferm]
427puppet-agent[20904]: (/Stage[main]/Base::Firewall/Ferm::Rule[monitoring-all]/File[/etc/ferm/conf.d/10_monitoring-all]/ensure) defined content as '{md5}925744ab53e15d5dfda3815b42cd3673'
428puppet-agent[20904]: (/Stage[main]/Base::Firewall/Ferm::Rule[monitoring-all]/File[/etc/ferm/conf.d/10_monitoring-all]) Scheduling refresh of Service[ferm]
429puppet-agent[20904]: (/Stage[main]/Base::Firewall/Ferm::Service[ssh-from-cumin-masters]/File[/etc/ferm/conf.d/10_ssh-from-cumin-masters]/ensure) defined content as '{md5}4ca567c959ee80ebc1349dbc5c63a2cc'
430puppet-agent[20904]: (/Stage[main]/Base::Firewall/Ferm::Service[ssh-from-cumin-masters]/File[/etc/ferm/conf.d/10_ssh-from-cumin-masters]) Scheduling refresh of Service[ferm]
431puppet-agent[20904]: (/Stage[main]/Base::Firewall/Sudo::User[nagios_check_ferm]/File[/etc/sudoers.d/nagios_check_ferm]/ensure) defined content as '{md5}f8cef3fc64ddcf933cf13ffaf787ab6c'
432puppet-agent[20904]: (/Stage[main]/Base::Firewall/Sudo::User[nagios_check_ferm]/File[/etc/sudoers.d/nagios_check_ferm]) Scheduling refresh of Exec[sudo_user_nagios_check_ferm_linting]
433puppet-agent[20904]: (/Stage[main]/Base::Firewall/Sudo::User[nagios_check_ferm]/Exec[sudo_user_nagios_check_ferm_linting]) Triggered 'refresh' from 1 events
434puppet-agent[20904]: (/Stage[main]/Profile::Webperf::Xhgui/Ferm::Service[webperf-xhgui-http]/File[/etc/ferm/conf.d/10_webperf-xhgui-http]/ensure) defined content as '{md5}261d14c6d49adcc1c32ef46d5b4904f8'
435puppet-agent[20904]: (/Stage[main]/Profile::Webperf::Xhgui/Ferm::Service[webperf-xhgui-http]/File[/etc/ferm/conf.d/10_webperf-xhgui-http]) Scheduling refresh of Service[ferm]
436puppet-agent[20904]: (/Stage[main]/Profile::Webperf::Xhgui/Ferm::Service[webperf-xhgui-mongo]/File[/etc/ferm/conf.d/10_webperf-xhgui-mongo]/ensure) defined content as '{md5}61e5f53dbc1c182a49a6a5b52c6a10a4'
437puppet-agent[20904]: (/Stage[main]/Profile::Webperf::Xhgui/Ferm::Service[webperf-xhgui-mongo]/File[/etc/ferm/conf.d/10_webperf-xhgui-mongo]) Scheduling refresh of Service[ferm]
438
439systemd[1]: Stopping ferm firewall configuration...
440kernel: [1201856.705372] ip6_tables: (C) 2000-2006 Netfilter Core Team
441ferm[29119]: Stopping Firewall: ferm.
442systemd[1]: Stopped ferm firewall configuration.
443systemd[1]: Starting ferm firewall configuration...
444ferm[29131]: Starting Firewall: ferm.
445systemd[1]: Started ferm firewall configuration.
446
447puppet-agent[20904]: (/Stage[main]/Ferm/Service[ferm]) Triggered 'refresh' from 16 events
448puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Conf[defaults]/File[/etc/apache2/conf-available/00-defaults.conf]/ensure) defined content as '{md5}cb4708de5aef2a292badee670a4386ef'
449puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Conf[defaults]/File[/etc/apache2/conf-available/00-defaults.conf]) Scheduling refresh of Service[apache2]
450puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Conf[defaults]/File[/etc/apache2/conf-enabled/00-defaults.conf]/ensure) created
451puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Conf[defaults]/File[/etc/apache2/conf-enabled/00-defaults.conf]) Scheduling refresh of Service[apache2]
452puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[authnz_ldap]/Exec[ensure_present_mod_authnz_ldap]/returns) executed successfully
453puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[authnz_ldap]/Exec[ensure_present_mod_authnz_ldap]) Scheduling refresh of Service[apache2]
454puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns) ERROR: Module mpm_event is enabled - cannot proceed due to conflicts. It needs to be disabled first!
455puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns) ERROR: Could not enable dependency mpm_prefork for php7.0, aborting
456puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns) Considering dependency mpm_prefork for php7.0:
457puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns) Considering conflict mpm_event for mpm_prefork:
458puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns) Considering conflict mpm_worker for mpm_prefork:
459puppet-agent[20904]: /usr/sbin/a2enmod php7.0 returned 1 instead of one of [0]
460puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns) change from notrun to 0 failed: /usr/sbin/a2enmod php7.0 returned 1 instead of one of [0]
461puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[rewrite]/Exec[ensure_present_mod_rewrite]/returns) executed successfully
462puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[rewrite]/Exec[ensure_present_mod_rewrite]) Scheduling refresh of Service[apache2]
463puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Conf[server-status]/File[/etc/apache2/conf-available/50-server-status.conf]/ensure) defined content as '{md5}d0e436f428fe56c243c2ab63582b7030'
464puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Conf[server-status]/File[/etc/apache2/conf-available/50-server-status.conf]) Scheduling refresh of Service[apache2]
465puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Conf[server-status]/File[/etc/apache2/conf-enabled/50-server-status.conf]/ensure) created
466puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Conf[server-status]/File[/etc/apache2/conf-enabled/50-server-status.conf]) Scheduling refresh of Service[apache2]
467puppet-agent[20904]: (/Stage[main]/Base::Firewall/Sysctl::Parameters[ferm_conntrack]/Sysctl::Conffile[ferm_conntrack]/File[/etc/sysctl.d/70-ferm_conntrack.conf]/ensure) defined content as '{md5}117ec150d2baac324d1e701edb170d3e'
468puppet-agent[20904]: (/Stage[main]/Base::Firewall/Sysctl::Parameters[ferm_conntrack]/Sysctl::Conffile[ferm_conntrack]/File[/etc/sysctl.d/70-ferm_conntrack.conf]) Scheduling refresh of Exec[update_sysctl]
469
470systemd[1]: Stopped Apply Kernel Variables.
471systemd[1]: Stopping Apply Kernel Variables...
472systemd[1]: Starting Apply Kernel Variables...
473systemd[1]: Started Apply Kernel Variables.
474
475puppet-agent[20904]: (/Stage[main]/Sysctl/Exec[update_sysctl]) Triggered 'refresh' from 1 events
476puppet-agent[20904]: (/Stage[main]/Base::Firewall/Nrpe::Monitor_service[conntrack_table_size]/Nrpe::Check[check_conntrack_table_size]/File[/etc/nagios/nrpe.d/check_conntrack_table_size.cfg]/ensure) defined content as '{md5}dea44d81c45749ddad7d2e74886c40bd'
477puppet-agent[20904]: (/Stage[main]/Base::Firewall/Nrpe::Monitor_service[conntrack_table_size]/Nrpe::Check[check_conntrack_table_size]/File[/etc/nagios/nrpe.d/check_conntrack_table_size.cfg]) Scheduling refresh of Service[nagios-nrpe-server]
478puppet-agent[20904]: (/Stage[main]/Base::Firewall/Nrpe::Monitor_service[ferm_active]/Nrpe::Check[check_ferm_active]/File[/etc/nagios/nrpe.d/check_ferm_active.cfg]/ensure) defined content as '{md5}8a9135b166fc29e735a6fd937deeb5b9'
479puppet-agent[20904]: (/Stage[main]/Base::Firewall/Nrpe::Monitor_service[ferm_active]/Nrpe::Check[check_ferm_active]/File[/etc/nagios/nrpe.d/check_ferm_active.cfg]) Scheduling refresh of Service[nagios-nrpe-server]
480
481systemd[1]: Stopping Nagios Remote Plugin Executor...
482nrpe[10371]: Caught SIGTERM - shutting down...
483systemd[1]: Stopped Nagios Remote Plugin Executor.
484nrpe[10371]: Daemon shutdown
485systemd[1]: Starting Nagios Remote Plugin Executor...
486systemd[1]: nagios-nrpe-server.service: PID file /var/run/nagios/nrpe.pid not readable (yet?) after start: No such file or directory
487nrpe[29208]: Starting up daemon
488systemd[1]: Started Nagios Remote Plugin Executor.
489
490puppet-agent[20904]: (/Stage[main]/Nrpe/Base::Service_unit[nagios-nrpe-server]/Service[nagios-nrpe-server]) Triggered 'refresh' from 2 events
491
492nrpe[29208]: Server listening on 10.68.17.129 port 5666.
493nrpe[29208]: Listening for connections on port 5666
494nrpe[29208]: Allowing connections from: 10.68.16.210
495
496puppet-agent[20904]: (/Stage[main]/Role::Webperf::Profiling_tools/System::Role[webperf::profiling_tools]/Motd::Script[role-webperf::profiling_tools]/File[/etc/update-motd.d/05-role-webperf--profiling-tools]/ensure) defined content as '{md5}daa468a681239dfacf68c4c8976885e7'
497puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Site[dummy]/Httpd::Conf[dummy]/File[/etc/apache2/sites-available/00-dummy.conf]/ensure) defined content as '{md5}6a7c5e68f0e8e576e9b751a61ba60239'
498puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Site[dummy]/Httpd::Conf[dummy]/File[/etc/apache2/sites-available/00-dummy.conf]) Scheduling refresh of Service[apache2]
499puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Site[dummy]/Httpd::Conf[dummy]/File[/etc/apache2/sites-enabled/00-dummy.conf]/ensure) created
500puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Site[dummy]/Httpd::Conf[dummy]/File[/etc/apache2/sites-enabled/00-dummy.conf]) Scheduling refresh of Service[apache2]
501puppet-agent[20904]: (/Stage[main]/Httpd/Service[apache2]) Dependency Exec[ensure_present_mod_php7.0] has failures: true
502puppet-agent[20904]: (/Stage[main]/Httpd/Service[apache2]) Skipping because of failed dependencies
503puppet-agent[20904]: (/Stage[main]/Httpd/Service[apache2]) Unscheduling all events on Service[apache2]
504puppet-agent[20904]: (Class[Httpd]) Unscheduling all events on Class[Httpd]
505puppet-agent[20904]: (Stage[main]) Unscheduling all events on Stage[main]
506puppet-agent[20904]: Applied catalog in 46.05 seconds

syslog at deployment-webperf12
puppet-agent[20904]: Using configured environment 'production' [..]
puppet-agent[20904]: Caching catalog for deployment-webperf12.deployment-prep.eqiad.wmflabs [..]
puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Conf[defaults]/File[/etc/apache2/conf-available/00-defaults.conf] [..]
puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[authnz_ldap]/Exec[ensure_present_mod_authnz_ldap]  [..]
puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns) ERROR: Module mpm_event is enabled - cannot proceed due to conflicts. It needs to be disabled first!
puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns) ERROR: Could not enable dependency mpm_prefork for php7.0, aborting
puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns) Considering dependency mpm_prefork for php7.0:
puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns) Considering conflict mpm_event for mpm_prefork:
puppet-agent[20904]: (/Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns) Considering conflict mpm_worker for mpm_prefork:
puppet-agent[20904]: /usr/sbin/a2enmod php7.0 returned 1 instead of one of [0]

[..]

puppet-agent[20904]: (/Stage[main]/Httpd/Service[apache2]) Dependency Exec[ensure_present_mod_php7.0] has failures: true
puppet-agent[20904]: (/Stage[main]/Httpd/Service[apache2]) Skipping because of failed dependencies
puppet-agent[20904]: (/Stage[main]/Httpd/Service[apache2]) Unscheduling all events on Service[apache2]
puppet-agent[20904]: (Class[Httpd]) Unscheduling all events on Class[Httpd]
puppet-agent[20904]: (Stage[main]) Unscheduling all events on Stage[main]

It looks puppet is mangling order of output:

Notice: /Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns: ERROR: Module mpm_event is enabled - cannot proceed due to conflicts. It needs to be disabled first!
Notice: /Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns: ERROR: Could not enable dependency mpm_prefork for php7.0, aborting
Notice: /Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns: Considering dependency mpm_prefork for php7.0:
Notice: /Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns: Considering conflict mpm_event for mpm_prefork:
Notice: /Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns: Considering conflict mpm_worker for mpm_prefork:
Error: /usr/sbin/a2enmod php7.0 returned 1 instead of one of [0]
Error: /Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns: change from notrun to 0 failed: /usr/sbin/a2enmod php7.0 returned 1 instead of one of [0]

becomes:

krenair@deployment-webperf12:~$ sudo a2enmod php7.0
Considering dependency mpm_prefork for php7.0:
Considering conflict mpm_event for mpm_prefork:
ERROR: Module mpm_event is enabled - cannot proceed due to conflicts. It needs to be disabled first!
Considering conflict mpm_worker for mpm_prefork:
ERROR: Could not enable dependency mpm_prefork for php7.0, aborting

/etc/apache2/mods-available/php7.0.load has Depends: mpm_prefork
But, /etc/apache2/mods-available/mpm_prefork.load has Conflicts: mpm_event
and that thing is enabled:

krenair@deployment-webperf12:~$ ls -lh /etc/apache2/mods-enabled/mpm_event.load 
lrwxrwxrwx 1 root root 32 Jul 17 01:38 /etc/apache2/mods-enabled/mpm_event.load -> ../mods-available/mpm_event.load

Now that /etc/apache2/mods-enabled/mpm_event.load file does not appear to be coming from puppet and dpkg -S does not show it coming from any package, so:

krenair@deployment-webperf12:~$ sudo rm /etc/apache2/mods-enabled/mpm_event.load
krenair@deployment-webperf12:~$ sudo puppet agent -tv
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for deployment-webperf12.deployment-prep.eqiad.wmflabs
Notice: /Stage[main]/Base::Environment/Tidy[/var/tmp/core]: Tidying 0 files
Info: Applying configuration version '1532196109'
Notice: /Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns: executed successfully
Info: /Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]: Scheduling refresh of Service[apache2]
Notice: /Stage[main]/Httpd/Service[apache2]: Triggered 'refresh' from 1 events
Notice: Applied catalog in 7.07 seconds

@Krenair Thanks, I’ll recreate it next week to try and rule out an issue with the manifest vs something we have manually done that maybe interfered.

Mentioned in SAL (#wikimedia-releng) [2018-07-23T15:34:15Z] <Krinkle> Deleting deployment-webperf12 - T195312
Mentioned in SAL (#wikimedia-releng) [2018-07-23T15:34:55Z] <Krinkle> Creating deployment-webperf13 - T195312
Mentioned in SAL (#wikimedia-releng) [2018-07-23T16:25:06Z] <Krinkle> Applying role::webperf::profiling_tools class to deployment-webperf13, T195312

It still failed, probably an issue with the manifest.

krinkle@deployment-webperf13:~$ sudo -i puppet agent --test --verbose
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for deployment-webperf13.deployment-prep.eqiad.wmflabs
Notice: /Stage[main]/Base::Environment/Tidy[/var/tmp/core]: Tidying 0 files
Info: Applying configuration version '1532363001'
Notice: Applied catalog in 6.71 seconds
krinkle@deployment-webperf13:~$ sudo -i puppet agent --test --verbose
(..)
Info: Caching catalog for deployment-webperf13.deployment-prep.eqiad.wmflabs
(..)
Notice: /Stage[main]/Role::Labs::Instance/Exec[enable_sites_local]/returns: executed successfully
Notice: /Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns: ERROR: Module mpm_event is enabled - cannot proceed due to conflicts. It needs to be disabled first!
Notice: /Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns: ERROR: Could not enable dependency mpm_prefork for php7.0, aborting
Notice: /Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns: Considering dependency mpm_prefork for php7.0:
Notice: /Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns: Considering conflict mpm_event for mpm_prefork:
Notice: /Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns: Considering conflict mpm_worker for mpm_prefork:
Error: /usr/sbin/a2enmod php7.0 returned 1 instead of one of [0]
Error: /Stage[main]/Httpd/Httpd::Mod_conf[php7.0]/Exec[ensure_present_mod_php7.0]/returns: change from notrun to 0 failed: /usr/sbin/a2enmod php7.0 returned 1 instead of one of [0]
Notice: /Stage[main]/Httpd/Service[apache2]: Dependency Exec[ensure_present_mod_php7.0] has failures: true
Warning: /Stage[main]/Httpd/Service[apache2]: Skipping because of failed dependencies
Info: Stage[main]: Unscheduling all events on Stage[main]

This is from DEBIAN/postinst in the apache2 package, and is called as part of the configure step:

enable_default_mpm()
{
        if is_fresh_install $@ ; then
                a2enmod -m -q mpm_event
        fi

}

https://github.com/wikimedia/puppet/blob/production/modules/httpd/manifests/mpm.pp#L31-L35 explicitly disables php5 if mpm_event is active, which seems worth noting (do we need to do the same for php7?), but doesn't solve the issue as such.

I think I know what's going on here: https://github.com/wikimedia/puppet/blob/production/modules/role/manifests/webperf/profiling_tools.pp#L17-L21

  • The require package on line 17 causes apache2 to be installed, because it's a dependency of libapache2-mod-php7.0.
  • When apache2 is installed as a dependency, it's configured to use mpm_event, as defined in the control file (see prior comment)
  • The next line causes apache to be configured to use php 7, but the mpm is never changed.

Probably the easiest option is to insert this on line 18:

class { '::httpd::mpm':
    mpm => 'worker'
}

(Or mpm => 'prefork', depending on context)

I'm not sure whether there's another option that's preferred in our environment, though.

On a related note, we might want to change the httpd::init class such that it detects the case where php[5|7] is being enabled, and will automatically call the httpd::mpm class.

Change 449367 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[operations/puppet@production] webperf: Move require_package for PHP from role to XHGui profile

https://gerrit.wikimedia.org/r/449367

Change 449367 merged by Dzahn:
[operations/puppet@production] webperf: Move require_package for PHP from role to XHGui profile

https://gerrit.wikimedia.org/r/449367

Mentioned in SAL (#wikimedia-releng) [2018-07-30T23:03:41Z] <Krinkle> Delete and recreate deployment-webperf13 (T195312 / T180761)

Mentioned in SAL (#wikimedia-releng) [2018-07-30T23:04:52Z] <Krinkle> Create instance deployment-webperf13 (deployment-webperf13 ) - T195312 / T180761

Mentioned in SAL (#wikimedia-releng) [2018-07-30T23:28:19Z] <Krinkle> Setting up puppet cert for deployment-webperf12; T195312 / T180761

Mentioned in SAL (#wikimedia-releng) [2018-07-31T17:50:49Z] <Krinkle> Apply role::webperf::profiling_tools to deployment-webperf12; T195312 / T180761

Change 449532 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[operations/puppet@production] webperf: Set mpm=worker explicitly for httpd.

https://gerrit.wikimedia.org/r/449532

Change 449532 merged by Dzahn:
[operations/puppet@production] webperf: Set mpm=prefork explicitly for profiling_tools' httpd

https://gerrit.wikimedia.org/r/449532