Page MenuHomePhabricator
Paste P14414

Journal between 2021-02-19 05:00:31 - 2021-02-19 05:10:31
ActivePublic

Authored by ProdPasteBot on Fri, Feb 19, 8:34 AM.
-- Logs begin at Tue 2020-11-10 03:39:48 UTC, end at Fri 2021-02-19 08:34:30 UTC. --
Feb 19 05:00:34 cloudvirt1024 sshd[4649]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:00:34 cloudvirt1024 sshd[4649]: Connection from 208.80.153.84 port 59398 on 10.64.20.43 port 22
Feb 19 05:00:34 cloudvirt1024 sshd[4649]: Connection closed by 208.80.153.84 port 59398 [preauth]
Feb 19 05:00:39 cloudvirt1024 sudo[4655]: nagios : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/lib/nagios/plugins/check_puppetrun -w 10800 -c 21600
Feb 19 05:00:39 cloudvirt1024 sudo[4655]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 19 05:00:39 cloudvirt1024 sudo[4655]: pam_unix(sudo:session): session closed for user root
Feb 19 05:01:01 cloudvirt1024 CRON[4664]: pam_unix(cron:session): session opened for user prometheus by (uid=0)
Feb 19 05:01:01 cloudvirt1024 CRON[4665]: (prometheus) CMD (/usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom)
Feb 19 05:01:01 cloudvirt1024 systemd[1]: Started Update Debian version stat exported by node_exporter.
Feb 19 05:01:01 cloudvirt1024 systemd[1]: Started Generate prometheus node metrics for cloudvirt ceph network usage.
Feb 19 05:01:01 cloudvirt1024 systemd[1]: prometheus-debian-version-textfile.service: Succeeded.
Feb 19 05:01:01 cloudvirt1024 prometheus-cloudvirt-ceph-network[4668]: [prometheus-cloudvirt-ceph-network] INFO: metrics have been dumped to /var/lib/prometheus/node.d/cloudvirt_ceph_network.prom
Feb 19 05:01:01 cloudvirt1024 systemd[1]: prometheus-node-cloudvirt-ceph-network.service: Succeeded.
Feb 19 05:01:01 cloudvirt1024 CRON[4664]: pam_unix(cron:session): session closed for user prometheus
Feb 19 05:01:31 cloudvirt1024 nova-compute[62835]: 2021-02-19 05:01:31.363 62835 WARNING nova.compute.resource_tracker [req-1492a22f-6b0d-44b0-b15f-6a174218312a - - - - -] Instance 98b912c0-8cf7-45f2-8728-942fdcdc803b has been moved to another host cloudvirt1022(cloudvirt1022.eqiad.wmnet). There are allocations remaining against the source host that might need to be removed: {'resources': {'VCPU': 1, 'MEMORY_MB': 2048, 'DISK_GB': 80}}.
Feb 19 05:01:37 cloudvirt1024 sshd[4774]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:01:37 cloudvirt1024 sshd[4774]: Connection from 208.80.154.88 port 53352 on 10.64.20.43 port 22
Feb 19 05:01:37 cloudvirt1024 sshd[4774]: Connection closed by 208.80.154.88 port 53352 [preauth]
Feb 19 05:01:58 cloudvirt1024 wmcs-backup[56558]: [37B blob data]
Feb 19 05:01:58 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:01:58,785] Cleaning up expired backups for VM wikidata-history-query-service.wdhqs-1(59e95aa5-ae70-47b7-9af8-f5c3f913a992)
Feb 19 05:01:59 cloudvirt1024 sshd[4880]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:01:59 cloudvirt1024 sshd[4880]: Connection from 208.80.153.84 port 38798 on 10.64.20.43 port 22
Feb 19 05:01:59 cloudvirt1024 sshd[4880]: Connection closed by 208.80.153.84 port 38798 [preauth]
Feb 19 05:02:01 cloudvirt1024 CRON[4882]: pam_unix(cron:session): session opened for user prometheus by (uid=0)
Feb 19 05:02:01 cloudvirt1024 CRON[4883]: (prometheus) CMD (/usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom)
Feb 19 05:02:01 cloudvirt1024 systemd[1]: Started Generate prometheus node metrics for cloudvirt ceph network usage.
Feb 19 05:02:01 cloudvirt1024 prometheus-cloudvirt-ceph-network[4885]: [prometheus-cloudvirt-ceph-network] INFO: metrics have been dumped to /var/lib/prometheus/node.d/cloudvirt_ceph_network.prom
Feb 19 05:02:01 cloudvirt1024 systemd[1]: prometheus-node-cloudvirt-ceph-network.service: Succeeded.
Feb 19 05:02:01 cloudvirt1024 CRON[4882]: pam_unix(cron:session): session closed for user prometheus
Feb 19 05:02:15 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:02:15,782] Cleaned up 1 expired backups for VM wikidata-history-query-service.wdhqs-1(59e95aa5-ae70-47b7-9af8-f5c3f913a992)
Feb 19 05:02:15 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:02:15,783] Creating backup for vm wikidata-realtime-dumps.wikidata-realtime-dumps(06179518-b478-4fdc-95a5-4b4e18b55208)
Feb 19 05:02:15 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:02:15,917] Creating differential backup of pool:eqiad1-compute image_name:06179518-b478-4fdc-95a5-4b4e18b55208_disk from rbd snapshot eqiad1-compute/06179518-b478-4fdc-95a5-4b4e18b55208_disk@2021-02-18T05:03:18_cloudvirt1024 and backy2 version 9e4c85ee-71a6-11eb-8679-b02628295df0
Feb 19 05:02:21 cloudvirt1024 sshd[5062]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:02:21 cloudvirt1024 sshd[5062]: Connection from 208.80.153.74 port 40348 on 10.64.20.43 port 22
Feb 19 05:02:21 cloudvirt1024 sshd[5062]: Connection closed by 208.80.153.74 port 40348 [preauth]
Feb 19 05:02:23 cloudvirt1024 sudo[5077]: nagios : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/sbin/ipmi-sensors -g Temperature,Power_Supply --quiet-cache --sdr-cache-recreate --interpret-oem-data --output-sensor-state --ignore-not-available-sensors --output-sensor-thresholds
Feb 19 05:02:23 cloudvirt1024 sudo[5077]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 19 05:02:23 cloudvirt1024 sudo[5077]: pam_unix(sudo:session): session closed for user root
Feb 19 05:02:33 cloudvirt1024 nova-compute[62835]: 2021-02-19 05:02:33.435 62835 WARNING nova.compute.resource_tracker [req-1492a22f-6b0d-44b0-b15f-6a174218312a - - - - -] Instance 98b912c0-8cf7-45f2-8728-942fdcdc803b has been moved to another host cloudvirt1022(cloudvirt1022.eqiad.wmnet). There are allocations remaining against the source host that might need to be removed: {'resources': {'VCPU': 1, 'MEMORY_MB': 2048, 'DISK_GB': 80}}.
Feb 19 05:02:36 cloudvirt1024 sshd[5135]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:02:36 cloudvirt1024 sshd[5135]: Connection from 208.80.154.84 port 47882 on 10.64.20.43 port 22
Feb 19 05:02:36 cloudvirt1024 sshd[5135]: Connection closed by 208.80.154.84 port 47882 [preauth]
Feb 19 05:02:40 cloudvirt1024 sudo[5151]: nagios : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/lib/nagios/plugins/check_puppetrun -w 10800 -c 21600
Feb 19 05:02:40 cloudvirt1024 sudo[5151]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 19 05:02:41 cloudvirt1024 sudo[5151]: pam_unix(sudo:session): session closed for user root
Feb 19 05:03:01 cloudvirt1024 CRON[5172]: pam_unix(cron:session): session opened for user prometheus by (uid=0)
Feb 19 05:03:01 cloudvirt1024 CRON[5173]: (prometheus) CMD (/usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom)
Feb 19 05:03:01 cloudvirt1024 systemd[1]: Started Update NIC firmware stats exported by node_exporter.
Feb 19 05:03:01 cloudvirt1024 systemd[1]: Started Generate prometheus node metrics for cloudvirt ceph network usage.
Feb 19 05:03:01 cloudvirt1024 systemd[1]: prometheus-nic-firmware-textfile.service: Succeeded.
Feb 19 05:03:01 cloudvirt1024 prometheus-cloudvirt-ceph-network[5175]: [prometheus-cloudvirt-ceph-network] INFO: metrics have been dumped to /var/lib/prometheus/node.d/cloudvirt_ceph_network.prom
Feb 19 05:03:01 cloudvirt1024 systemd[1]: prometheus-node-cloudvirt-ceph-network.service: Succeeded.
Feb 19 05:03:01 cloudvirt1024 CRON[5172]: pam_unix(cron:session): session closed for user prometheus
Feb 19 05:03:19 cloudvirt1024 sshd[5220]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:03:19 cloudvirt1024 sshd[5220]: Connection from 208.80.154.88 port 38290 on 10.64.20.43 port 22
Feb 19 05:03:19 cloudvirt1024 sshd[5220]: Connection closed by 208.80.154.88 port 38290 [preauth]
Feb 19 05:03:27 cloudvirt1024 neutron-linuxbridge-agent[57204]: 2021-02-19 05:03:27.896 57204 INFO neutron.agent.securitygroups_rpc [req-91c2cfc0-50e6-485f-b541-04bcb8439c79 osstackcanary admin-monitoring - - -] Security group member updated ['85904753-5054-4af3-b8fc-9ad63fe4d765']
Feb 19 05:03:28 cloudvirt1024 sshd[5240]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:03:28 cloudvirt1024 sshd[5240]: Connection from 208.80.153.84 port 48064 on 10.64.20.43 port 22
Feb 19 05:03:28 cloudvirt1024 sshd[5240]: Connection closed by 208.80.153.84 port 48064 [preauth]
Feb 19 05:03:33 cloudvirt1024 nova-compute[62835]: 2021-02-19 05:03:33.441 62835 WARNING nova.compute.resource_tracker [req-1492a22f-6b0d-44b0-b15f-6a174218312a - - - - -] Instance 98b912c0-8cf7-45f2-8728-942fdcdc803b has been moved to another host cloudvirt1022(cloudvirt1022.eqiad.wmnet). There are allocations remaining against the source host that might need to be removed: {'resources': {'VCPU': 1, 'MEMORY_MB': 2048, 'DISK_GB': 80}}.
Feb 19 05:03:59 cloudvirt1024 sshd[5338]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:03:59 cloudvirt1024 sshd[5338]: Connection from 208.80.153.74 port 49842 on 10.64.20.43 port 22
Feb 19 05:03:59 cloudvirt1024 sshd[5338]: Connection closed by 208.80.153.74 port 49842 [preauth]
Feb 19 05:04:01 cloudvirt1024 CRON[5346]: pam_unix(cron:session): session opened for user prometheus by (uid=0)
Feb 19 05:04:01 cloudvirt1024 CRON[5347]: (prometheus) CMD (/usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom)
Feb 19 05:04:01 cloudvirt1024 systemd[1]: Started Generate prometheus node metrics for cloudvirt ceph network usage.
Feb 19 05:04:01 cloudvirt1024 prometheus-cloudvirt-ceph-network[5349]: [prometheus-cloudvirt-ceph-network] INFO: metrics have been dumped to /var/lib/prometheus/node.d/cloudvirt_ceph_network.prom
Feb 19 05:04:01 cloudvirt1024 systemd[1]: prometheus-node-cloudvirt-ceph-network.service: Succeeded.
Feb 19 05:04:01 cloudvirt1024 CRON[5346]: pam_unix(cron:session): session closed for user prometheus
Feb 19 05:04:08 cloudvirt1024 sudo[5363]: nagios : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/lib/nagios/plugins/check_puppetrun -w 10800 -c 21600
Feb 19 05:04:08 cloudvirt1024 sudo[5363]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 19 05:04:08 cloudvirt1024 sudo[5363]: pam_unix(sudo:session): session closed for user root
Feb 19 05:04:32 cloudvirt1024 sshd[5396]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:04:32 cloudvirt1024 sshd[5396]: Connection from 208.80.154.84 port 59104 on 10.64.20.43 port 22
Feb 19 05:04:32 cloudvirt1024 sshd[5396]: Connection closed by 208.80.154.84 port 59104 [preauth]
Feb 19 05:04:35 cloudvirt1024 nova-compute[62835]: 2021-02-19 05:04:35.463 62835 WARNING nova.compute.resource_tracker [req-1492a22f-6b0d-44b0-b15f-6a174218312a - - - - -] Instance 98b912c0-8cf7-45f2-8728-942fdcdc803b has been moved to another host cloudvirt1022(cloudvirt1022.eqiad.wmnet). There are allocations remaining against the source host that might need to be removed: {'resources': {'VCPU': 1, 'MEMORY_MB': 2048, 'DISK_GB': 80}}.
Feb 19 05:04:36 cloudvirt1024 dbus-daemon[1204]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service' requested by ':1.8234388' (uid=109 pid=5481 comm="timedatectl show ")
Feb 19 05:04:36 cloudvirt1024 systemd[1]: Starting Time & Date Service...
Feb 19 05:04:36 cloudvirt1024 dbus-daemon[1204]: [system] Successfully activated service 'org.freedesktop.timedate1'
Feb 19 05:04:36 cloudvirt1024 systemd[1]: Started Time & Date Service.
Feb 19 05:04:38 cloudvirt1024 sudo[5511]: nagios : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/lib/nagios/plugins/check_long_procs -w 240 -c 480
Feb 19 05:04:38 cloudvirt1024 sudo[5511]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 19 05:04:38 cloudvirt1024 sudo[5511]: pam_unix(sudo:session): session closed for user root
Feb 19 05:05:00 cloudvirt1024 systemd[1]: Started Generate prometheus node metrics for cloudvirt ceph network usage.
Feb 19 05:05:00 cloudvirt1024 systemd[1]: Started reconciliation job in case any debmonitor update fails.
Feb 19 05:05:00 cloudvirt1024 prometheus-cloudvirt-ceph-network[5595]: [prometheus-cloudvirt-ceph-network] INFO: metrics have been dumped to /var/lib/prometheus/node.d/cloudvirt_ceph_network.prom
Feb 19 05:05:00 cloudvirt1024 systemd[1]: prometheus-node-cloudvirt-ceph-network.service: Succeeded.
Feb 19 05:05:01 cloudvirt1024 debmonitor-client[5596]: INFO:debmonitor:Found 1075 installed binary packages
Feb 19 05:05:01 cloudvirt1024 sshd[5606]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:05:01 cloudvirt1024 sshd[5606]: Connection from 208.80.154.88 port 51840 on 10.64.20.43 port 22
Feb 19 05:05:01 cloudvirt1024 sshd[5606]: Connection closed by 208.80.154.88 port 51840 [preauth]
Feb 19 05:05:01 cloudvirt1024 CRON[5608]: pam_unix(cron:session): session opened for user root by (uid=0)
Feb 19 05:05:01 cloudvirt1024 CRON[5609]: pam_unix(cron:session): session opened for user prometheus by (uid=0)
Feb 19 05:05:01 cloudvirt1024 CRON[5610]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Feb 19 05:05:01 cloudvirt1024 CRON[5611]: (prometheus) CMD (/usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom)
Feb 19 05:05:01 cloudvirt1024 CRON[5608]: pam_unix(cron:session): session closed for user root
Feb 19 05:05:01 cloudvirt1024 CRON[5609]: pam_unix(cron:session): session closed for user prometheus
Feb 19 05:05:01 cloudvirt1024 debmonitor-client[5596]: INFO:debmonitor:Found 14 upgradable binary packages (including new dependencies)
Feb 19 05:05:01 cloudvirt1024 sshd[5618]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:05:01 cloudvirt1024 sshd[5618]: Connection from 208.80.153.84 port 59566 on 10.64.20.43 port 22
Feb 19 05:05:01 cloudvirt1024 sshd[5618]: Connection closed by 208.80.153.84 port 59566 [preauth]
Feb 19 05:05:02 cloudvirt1024 debmonitor-client[5596]: INFO:debmonitor:Successfully sent the full update to the DebMonitor server
Feb 19 05:05:02 cloudvirt1024 systemd[1]: debmonitor-client.service: Succeeded.
Feb 19 05:05:19 cloudvirt1024 wmcs-backup[56558]: [37B blob data]
Feb 19 05:05:19 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:05:19,905] Cleaning up expired backups for VM wikidata-realtime-dumps.wikidata-realtime-dumps(06179518-b478-4fdc-95a5-4b4e18b55208)
Feb 19 05:05:24 cloudvirt1024 systemd[1]: systemd-timedated.service: Succeeded.
Feb 19 05:05:28 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:05:28,458] Cleaned up 1 expired backups for VM wikidata-realtime-dumps.wikidata-realtime-dumps(06179518-b478-4fdc-95a5-4b4e18b55208)
Feb 19 05:05:28 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:05:28,459] Creating backup for vm wikidocumentaries.roope(9dd91776-e93e-4f56-88cb-ce355f918076)
Feb 19 05:05:28 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:05:28,621] Creating differential backup of pool:eqiad1-compute image_name:9dd91776-e93e-4f56-88cb-ce355f918076_disk from rbd snapshot eqiad1-compute/9dd91776-e93e-4f56-88cb-ce355f918076_disk@2021-02-18T05:06:31_cloudvirt1024 and backy2 version 1102c4b8-71a7-11eb-bda1-b02628295df0
Feb 19 05:05:36 cloudvirt1024 nova-compute[62835]: 2021-02-19 05:05:36.483 62835 WARNING nova.compute.resource_tracker [req-1492a22f-6b0d-44b0-b15f-6a174218312a - - - - -] Instance 98b912c0-8cf7-45f2-8728-942fdcdc803b has been moved to another host cloudvirt1022(cloudvirt1022.eqiad.wmnet). There are allocations remaining against the source host that might need to be removed: {'resources': {'VCPU': 1, 'MEMORY_MB': 2048, 'DISK_GB': 80}}.
Feb 19 05:05:41 cloudvirt1024 sshd[5943]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:05:41 cloudvirt1024 sshd[5943]: Connection from 208.80.153.74 port 33408 on 10.64.20.43 port 22
Feb 19 05:05:41 cloudvirt1024 sshd[5943]: Connection closed by 208.80.153.74 port 33408 [preauth]
Feb 19 05:05:52 cloudvirt1024 sudo[5974]: nagios : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/lib/nagios/plugins/check_puppetrun -w 10800 -c 21600
Feb 19 05:05:52 cloudvirt1024 sudo[5974]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 19 05:05:52 cloudvirt1024 sudo[5974]: pam_unix(sudo:session): session closed for user root
Feb 19 05:05:53 cloudvirt1024 wmcs-backup[56558]: [37B blob data]
Feb 19 05:05:53 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:05:53,497] Cleaning up expired backups for VM wikidocumentaries.roope(9dd91776-e93e-4f56-88cb-ce355f918076)
Feb 19 05:05:56 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:05:56,527] Cleaned up 1 expired backups for VM wikidocumentaries.roope(9dd91776-e93e-4f56-88cb-ce355f918076)
Feb 19 05:05:56 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:05:56,527] Creating backup for vm wikidocumentaries.hupu(b6305f09-d892-4122-a08e-bb00d0cd925f)
Feb 19 05:05:56 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:05:56,693] Creating differential backup of pool:eqiad1-compute image_name:b6305f09-d892-4122-a08e-bb00d0cd925f_disk from rbd snapshot eqiad1-compute/b6305f09-d892-4122-a08e-bb00d0cd925f_disk@2021-02-18T05:06:59_cloudvirt1024 and backy2 version 21d6b06a-71a7-11eb-bc24-b02628295df0
Feb 19 05:06:01 cloudvirt1024 CRON[6224]: pam_unix(cron:session): session opened for user prometheus by (uid=0)
Feb 19 05:06:01 cloudvirt1024 CRON[6225]: (prometheus) CMD (/usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom)
Feb 19 05:06:01 cloudvirt1024 systemd[1]: Started Update Debian version stat exported by node_exporter.
Feb 19 05:06:01 cloudvirt1024 systemd[1]: Started Generate prometheus node metrics for cloudvirt ceph network usage.
Feb 19 05:06:01 cloudvirt1024 systemd[1]: prometheus-debian-version-textfile.service: Succeeded.
Feb 19 05:06:01 cloudvirt1024 prometheus-cloudvirt-ceph-network[6228]: [prometheus-cloudvirt-ceph-network] INFO: metrics have been dumped to /var/lib/prometheus/node.d/cloudvirt_ceph_network.prom
Feb 19 05:06:01 cloudvirt1024 systemd[1]: prometheus-node-cloudvirt-ceph-network.service: Succeeded.
Feb 19 05:06:01 cloudvirt1024 CRON[6224]: pam_unix(cron:session): session closed for user prometheus
Feb 19 05:06:35 cloudvirt1024 sshd[6293]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:06:35 cloudvirt1024 sshd[6293]: Connection from 208.80.154.84 port 45170 on 10.64.20.43 port 22
Feb 19 05:06:35 cloudvirt1024 sshd[6293]: Connection closed by 208.80.154.84 port 45170 [preauth]
Feb 19 05:06:36 cloudvirt1024 nova-compute[62835]: 2021-02-19 05:06:36.257 62835 WARNING nova.compute.resource_tracker [req-1492a22f-6b0d-44b0-b15f-6a174218312a - - - - -] Instance 98b912c0-8cf7-45f2-8728-942fdcdc803b has been moved to another host cloudvirt1022(cloudvirt1022.eqiad.wmnet). There are allocations remaining against the source host that might need to be removed: {'resources': {'VCPU': 1, 'MEMORY_MB': 2048, 'DISK_GB': 80}}.
Feb 19 05:06:41 cloudvirt1024 sshd[6325]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:06:41 cloudvirt1024 sshd[6325]: Connection from 208.80.153.84 port 44870 on 10.64.20.43 port 22
Feb 19 05:06:41 cloudvirt1024 sshd[6325]: Connection closed by 208.80.153.84 port 44870 [preauth]
Feb 19 05:06:41 cloudvirt1024 sudo[6331]: nagios : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/lib/nagios/plugins/check_puppetrun -w 10800 -c 21600
Feb 19 05:06:41 cloudvirt1024 sudo[6331]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 19 05:06:42 cloudvirt1024 sudo[6331]: pam_unix(sudo:session): session closed for user root
Feb 19 05:06:42 cloudvirt1024 sshd[6334]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:06:42 cloudvirt1024 sshd[6334]: Connection from 208.80.154.88 port 36662 on 10.64.20.43 port 22
Feb 19 05:06:42 cloudvirt1024 sshd[6334]: Connection closed by 208.80.154.88 port 36662 [preauth]
Feb 19 05:06:58 cloudvirt1024 wmcs-backup[56558]: [37B blob data]
Feb 19 05:06:59 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:06:59,342] Cleaning up expired backups for VM wikidocumentaries.hupu(b6305f09-d892-4122-a08e-bb00d0cd925f)
Feb 19 05:06:59 cloudvirt1024 sudo[6424]: nagios : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/lib/nagios/plugins/check_raid megacli
Feb 19 05:06:59 cloudvirt1024 sudo[6424]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 19 05:07:00 cloudvirt1024 sudo[6424]: pam_unix(sudo:session): session closed for user root
Feb 19 05:07:00 cloudvirt1024 systemd[1]: Started Generate prometheus node metrics for cloudvirt ceph network usage.
Feb 19 05:07:00 cloudvirt1024 prometheus-cloudvirt-ceph-network[6428]: [prometheus-cloudvirt-ceph-network] INFO: metrics have been dumped to /var/lib/prometheus/node.d/cloudvirt_ceph_network.prom
Feb 19 05:07:00 cloudvirt1024 systemd[1]: prometheus-node-cloudvirt-ceph-network.service: Succeeded.
Feb 19 05:07:01 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:07:01,442] Cleaned up 1 expired backups for VM wikidocumentaries.hupu(b6305f09-d892-4122-a08e-bb00d0cd925f)
Feb 19 05:07:01 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:07:01,443] Creating backup for vm wikilabels.wikilabels-backups-01(4a85bfa1-f920-489c-badb-e8cc9f8d5692)
Feb 19 05:07:01 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:07:01,578] Creating differential backup of pool:eqiad1-compute image_name:4a85bfa1-f920-489c-badb-e8cc9f8d5692_disk from rbd snapshot eqiad1-compute/4a85bfa1-f920-489c-badb-e8cc9f8d5692_disk@2021-02-18T05:08:06_cloudvirt1024 and backy2 version 49a827b8-71a7-11eb-9cd6-b02628295df0
Feb 19 05:07:01 cloudvirt1024 CRON[6574]: pam_unix(cron:session): session opened for user prometheus by (uid=0)
Feb 19 05:07:01 cloudvirt1024 CRON[6575]: (prometheus) CMD (/usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom)
Feb 19 05:07:02 cloudvirt1024 CRON[6574]: pam_unix(cron:session): session closed for user prometheus
Feb 19 05:07:14 cloudvirt1024 wmcs-backup[56558]: [37B blob data]
Feb 19 05:07:15 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:07:15,139] Cleaning up expired backups for VM wikilabels.wikilabels-backups-01(4a85bfa1-f920-489c-badb-e8cc9f8d5692)
Feb 19 05:07:16 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:07:16,624] Cleaned up 1 expired backups for VM wikilabels.wikilabels-backups-01(4a85bfa1-f920-489c-badb-e8cc9f8d5692)
Feb 19 05:07:16 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:07:16,625] Creating backup for vm wikilabels.wikilabels-staging-02(65343972-d77a-4c93-a69f-6a0ff15d082c)
Feb 19 05:07:16 cloudvirt1024 wmcs-backup[56558]: INFO:[2021-02-19 05:07:16,773] Creating differential backup of pool:eqiad1-compute image_name:65343972-d77a-4c93-a69f-6a0ff15d082c_disk from rbd snapshot eqiad1-compute/65343972-d77a-4c93-a69f-6a0ff15d082c_disk@2021-02-18T05:08:24_cloudvirt1024 and backy2 version 547f81fe-71a7-11eb-bee1-b02628295df0
Feb 19 05:07:26 cloudvirt1024 sshd[6916]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:07:26 cloudvirt1024 sshd[6916]: Connection from 208.80.153.74 port 45728 on 10.64.20.43 port 22
Feb 19 05:07:26 cloudvirt1024 sshd[6916]: Connection closed by 208.80.153.74 port 45728 [preauth]
Feb 19 05:07:27 cloudvirt1024 sudo[6922]: nagios : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/lib/nagios/plugins/check_raid megacli
Feb 19 05:07:27 cloudvirt1024 sudo[6922]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 19 05:07:27 cloudvirt1024 sudo[6922]: pam_unix(sudo:session): session closed for user root
Feb 19 05:07:31 cloudvirt1024 kernel: safe_timer[6867]: segfault at 7fdcad21a728 ip 00007fdc9777ebcf sp 00007fdc867fb970 error 4 in libceph-common.so.0[7fdc975d2000+5e8000]
Feb 19 05:07:31 cloudvirt1024 kernel: Code: 1f 40 00 4d 89 f4 eb bb 0f 1f 00 41 57 41 56 4c 8d 77 08 41 55 41 54 55 53 48 83 ec 08 48 8b 5f 10 48 85 db 0f 84 d9 00 00 00 <4c> 8b 7e 08 4c 8b 2e 4d 89 f4 eb 11 0f 1f 44 00 00 49 89 dc 48 8b
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: Traceback (most recent call last):
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: File "/usr/local/sbin/wmcs-backup", line 2012, in <module>
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: args.func()
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: File "/usr/local/sbin/wmcs-backup", line 1847, in <lambda>
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: ).backup_assigned_vms(from_cache=args.from_cache, noop=args.noop)
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: File "/usr/local/sbin/wmcs-backup", line 1312, in backup_assigned_vms
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: noop=noop,
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: File "/usr/local/sbin/wmcs-backup", line 1121, in create_vm_backup
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: vm_info=vm_info, noop=noop
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: File "/usr/local/sbin/wmcs-backup", line 1030, in create_vm_backup
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: new_backup = self.vms_backups[vm_id].create_next_backup(noop=noop)
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: File "/usr/local/sbin/wmcs-backup", line 720, in create_next_backup
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: noop=noop,
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: File "/usr/local/sbin/wmcs-backup", line 551, in create_diff_backup
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: noop=noop,
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: File "/usr/lib/python3/dist-packages/rbd2backy2.py", line 332, in create_diff_backup
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: noop=noop,
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: File "/usr/lib/python3/dist-packages/rbd2backy2.py", line 31, in run_command
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: output = subprocess.check_output(args).decode("utf8")
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: File "/usr/lib/python3.7/subprocess.py", line 395, in check_output
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: **kwargs).stdout
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: File "/usr/lib/python3.7/subprocess.py", line 487, in run
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: output=stdout, stderr=stderr)
Feb 19 05:07:31 cloudvirt1024 wmcs-backup[56558]: subprocess.CalledProcessError: Command '['/usr/bin/backy2', 'backup', '--snapshot-name', '2021-02-19T05:07:16_cloudvirt1024', '--rbd', '/tmp/tmpbgykl78e', '--from-version', '547f81fe-71a7-11eb-bee1-b02628295df0', 'rbd://eqiad1-compute/65343972-d77a-4c93-a69f-6a0ff15d082c_disk@2021-02-19T05:07:16_cloudvirt1024', '65343972-d77a-4c93-a69f-6a0ff15d082c_disk', '--expire', '2021-02-24 05:07:16', '--tag', 'differential_backup']' died with <Signals.SIGSEGV: 11>.
Feb 19 05:07:31 cloudvirt1024 systemd[1]: backup_vms.service: Main process exited, code=exited, status=1/FAILURE
Feb 19 05:07:31 cloudvirt1024 systemd[1]: backup_vms.service: Failed with result 'exit-code'.
Feb 19 05:07:36 cloudvirt1024 nova-compute[62835]: 2021-02-19 05:07:36.410 62835 WARNING nova.compute.resource_tracker [req-1492a22f-6b0d-44b0-b15f-6a174218312a - - - - -] Instance 98b912c0-8cf7-45f2-8728-942fdcdc803b has been moved to another host cloudvirt1022(cloudvirt1022.eqiad.wmnet). There are allocations remaining against the source host that might need to be removed: {'resources': {'VCPU': 1, 'MEMORY_MB': 2048, 'DISK_GB': 80}}.
Feb 19 05:08:01 cloudvirt1024 CRON[7045]: pam_unix(cron:session): session opened for user root by (uid=0)
Feb 19 05:08:01 cloudvirt1024 CRON[7046]: pam_unix(cron:session): session opened for user prometheus by (uid=0)
Feb 19 05:08:01 cloudvirt1024 CRON[7048]: (root) CMD (/usr/local/sbin/puppet-run > /dev/null 2>&1)
Feb 19 05:08:01 cloudvirt1024 CRON[7047]: (prometheus) CMD (/usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom)
Feb 19 05:08:01 cloudvirt1024 systemd[1]: Started Generate prometheus node metrics for cloudvirt ceph network usage.
Feb 19 05:08:01 cloudvirt1024 prometheus-cloudvirt-ceph-network[7051]: [prometheus-cloudvirt-ceph-network] INFO: metrics have been dumped to /var/lib/prometheus/node.d/cloudvirt_ceph_network.prom
Feb 19 05:08:01 cloudvirt1024 systemd[1]: prometheus-node-cloudvirt-ceph-network.service: Succeeded.
Feb 19 05:08:01 cloudvirt1024 CRON[7046]: pam_unix(cron:session): session closed for user prometheus
Feb 19 05:08:01 cloudvirt1024 puppet-agent-cronjob[7065]: Sleeping 1 for random splay
Feb 19 05:08:04 cloudvirt1024 su[7785]: (to debmonitor) root on none
Feb 19 05:08:04 cloudvirt1024 su[7785]: pam_unix(su-l:session): session opened for user debmonitor by (uid=0)
Feb 19 05:08:04 cloudvirt1024 systemd[1]: Started Update NIC firmware stats exported by node_exporter.
Feb 19 05:08:04 cloudvirt1024 systemd[1]: Created slice User Slice of UID 998.
Feb 19 05:08:04 cloudvirt1024 systemd[1]: Starting User Runtime Directory /run/user/998...
Feb 19 05:08:04 cloudvirt1024 systemd[1]: Started User Runtime Directory /run/user/998.
Feb 19 05:08:04 cloudvirt1024 systemd[1]: Starting User Manager for UID 998...
Feb 19 05:08:04 cloudvirt1024 systemd[1]: prometheus-nic-firmware-textfile.service: Succeeded.
Feb 19 05:08:04 cloudvirt1024 systemd[7807]: pam_unix(systemd-user:session): session opened for user debmonitor by (uid=0)
Feb 19 05:08:04 cloudvirt1024 systemd[7807]: gpgconf: running /usr/bin/gpg-agent failed (exitcode=2): General error
Feb 19 05:08:04 cloudvirt1024 systemd[7807]: gpgconf: fatal error (exit status 1)
Feb 19 05:08:04 cloudvirt1024 systemd[7807]: Listening on GnuPG cryptographic agent (ssh-agent emulation).
Feb 19 05:08:04 cloudvirt1024 systemd[7807]: Listening on GnuPG cryptographic agent and passphrase cache (restricted).
Feb 19 05:08:04 cloudvirt1024 systemd[7807]: Listening on GnuPG cryptographic agent and passphrase cache.
Feb 19 05:08:04 cloudvirt1024 systemd[7807]: Listening on GnuPG cryptographic agent and passphrase cache (access for web browsers).
Feb 19 05:08:04 cloudvirt1024 systemd[7807]: Reached target Timers.
Feb 19 05:08:04 cloudvirt1024 systemd[7807]: Reached target Paths.
Feb 19 05:08:04 cloudvirt1024 systemd[7807]: Listening on GnuPG network certificate management daemon.
Feb 19 05:08:04 cloudvirt1024 systemd[7807]: Reached target Sockets.
Feb 19 05:08:04 cloudvirt1024 systemd[7807]: Reached target Basic System.
Feb 19 05:08:04 cloudvirt1024 systemd[7807]: Reached target Default.
Feb 19 05:08:04 cloudvirt1024 systemd[7807]: Startup finished in 46ms.
Feb 19 05:08:04 cloudvirt1024 systemd[1]: Started User Manager for UID 998.
Feb 19 05:08:04 cloudvirt1024 systemd[1]: Started Session 242530 of user debmonitor.
Feb 19 05:08:04 cloudvirt1024 puppet-agent-cronjob[7068]: su: warning: cannot change directory to /nonexistent: No such file or directory
Feb 19 05:08:06 cloudvirt1024 puppet-agent-cronjob[7068]: INFO:debmonitor:Found 1075 installed binary packages
Feb 19 05:08:06 cloudvirt1024 puppet-agent-cronjob[7068]: INFO:debmonitor:Found 14 upgradable binary packages (including new dependencies)
Feb 19 05:08:07 cloudvirt1024 puppet-agent-cronjob[7068]: INFO:debmonitor:Successfully sent the upgradable update to the DebMonitor server
Feb 19 05:08:07 cloudvirt1024 su[7785]: pam_unix(su-l:session): session closed for user debmonitor
Feb 19 05:08:07 cloudvirt1024 systemd[1]: session-242530.scope: Succeeded.
Feb 19 05:08:08 cloudvirt1024 puppet-agent-cronjob[7068]: W: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'http://mirrors.wikimedia.org/osbpo buster-train-backports-nochange InRelease' does not seem to provide it (sources.list entry misspelt?)
Feb 19 05:08:08 cloudvirt1024 puppet-agent-cronjob[7068]: W: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'http://mirrors.wikimedia.org/osbpo buster-train-backports InRelease' does not seem to provide it (sources.list entry misspelt?)
Feb 19 05:08:10 cloudvirt1024 puppet-agent[7829]: Using configured environment 'production'
Feb 19 05:08:10 cloudvirt1024 puppet-agent[7829]: Retrieving pluginfacts
Feb 19 05:08:10 cloudvirt1024 puppet-agent[7829]: Retrieving plugin
Feb 19 05:08:10 cloudvirt1024 puppet-agent[7829]: Retrieving locales
Feb 19 05:08:10 cloudvirt1024 puppet-agent[7829]: Loading facts
Feb 19 05:08:17 cloudvirt1024 systemd[1]: Stopping User Manager for UID 998...
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: Stopped target Default.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: Stopped target Basic System.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: Stopped target Paths.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: Stopped target Sockets.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: gpg-agent-extra.socket: Succeeded.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: Closed GnuPG cryptographic agent and passphrase cache (restricted).
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: Stopped target Timers.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: gpg-agent-ssh.socket: Succeeded.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: Closed GnuPG cryptographic agent (ssh-agent emulation).
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: gpg-agent.socket: Succeeded.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: Closed GnuPG cryptographic agent and passphrase cache.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: dirmngr.socket: Succeeded.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: Closed GnuPG network certificate management daemon.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: gpg-agent-browser.socket: Succeeded.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: Closed GnuPG cryptographic agent and passphrase cache (access for web browsers).
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: Reached target Shutdown.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: systemd-exit.service: Succeeded.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: Started Exit the Session.
Feb 19 05:08:17 cloudvirt1024 systemd[7807]: Reached target Exit the Session.
Feb 19 05:08:17 cloudvirt1024 systemd[1]: user@998.service: Succeeded.
Feb 19 05:08:17 cloudvirt1024 systemd[1]: Stopped User Manager for UID 998.
Feb 19 05:08:17 cloudvirt1024 systemd[1]: Stopping User Runtime Directory /run/user/998...
Feb 19 05:08:17 cloudvirt1024 systemd[1]: run-user-998.mount: Succeeded.
Feb 19 05:08:17 cloudvirt1024 systemd[1]: user-runtime-dir@998.service: Succeeded.
Feb 19 05:08:17 cloudvirt1024 systemd[1]: Stopped User Runtime Directory /run/user/998.
Feb 19 05:08:17 cloudvirt1024 systemd[1]: Removed slice User Slice of UID 998.
Feb 19 05:08:17 cloudvirt1024 sshd[7945]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:08:17 cloudvirt1024 sshd[7945]: Connection from 208.80.153.84 port 57602 on 10.64.20.43 port 22
Feb 19 05:08:17 cloudvirt1024 sshd[7945]: Connection closed by 208.80.153.84 port 57602 [preauth]
Feb 19 05:08:20 cloudvirt1024 puppet-agent[7829]: Caching catalog for cloudvirt1024.eqiad.wmnet
Feb 19 05:08:20 cloudvirt1024 puppet-agent[7829]: Applying configuration version '(23122da30b) Dzahn - Revert "scap: remove mwmaint2001 from "dsh" groups"'
Feb 19 05:08:22 cloudvirt1024 sudo[7956]: nagios : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/lib/nagios/plugins/check_puppetrun -w 10800 -c 21600
Feb 19 05:08:22 cloudvirt1024 sudo[7956]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 19 05:08:22 cloudvirt1024 sudo[7956]: pam_unix(sudo:session): session closed for user root
Feb 19 05:08:22 cloudvirt1024 crontab[7959]: (root) LIST (root)
Feb 19 05:08:22 cloudvirt1024 crontab[7962]: (root) LIST (prometheus)
Feb 19 05:08:22 cloudvirt1024 crontab[7965]: (root) LIST (debmonitor)
Feb 19 05:08:25 cloudvirt1024 sshd[8052]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:08:25 cloudvirt1024 sshd[8052]: Connection from 208.80.154.88 port 49924 on 10.64.20.43 port 22
Feb 19 05:08:25 cloudvirt1024 sshd[8052]: Connection closed by 208.80.154.88 port 49924 [preauth]
Feb 19 05:08:32 cloudvirt1024 sudo[8264]: nagios : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/lib/nagios/plugins/check_raid megacli
Feb 19 05:08:32 cloudvirt1024 sudo[8264]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 19 05:08:32 cloudvirt1024 sudo[8264]: pam_unix(sudo:session): session closed for user root
Feb 19 05:08:36 cloudvirt1024 sshd[9134]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:08:36 cloudvirt1024 sshd[9134]: Connection from 208.80.154.84 port 58436 on 10.64.20.43 port 22
Feb 19 05:08:36 cloudvirt1024 sshd[9134]: Connection closed by 208.80.154.84 port 58436 [preauth]
Feb 19 05:08:37 cloudvirt1024 puppet-agent[7829]: Applied catalog in 17.41 seconds
Feb 19 05:08:38 cloudvirt1024 nova-compute[62835]: 2021-02-19 05:08:38.241 62835 WARNING nova.compute.resource_tracker [req-1492a22f-6b0d-44b0-b15f-6a174218312a - - - - -] Instance 98b912c0-8cf7-45f2-8728-942fdcdc803b has been moved to another host cloudvirt1022(cloudvirt1022.eqiad.wmnet). There are allocations remaining against the source host that might need to be removed: {'resources': {'VCPU': 1, 'MEMORY_MB': 2048, 'DISK_GB': 80}}.
Feb 19 05:08:38 cloudvirt1024 CRON[7045]: pam_unix(cron:session): session closed for user root
Feb 19 05:08:53 cloudvirt1024 neutron-linuxbridge-agent[57204]: 2021-02-19 05:08:53.774 57204 INFO neutron.agent.securitygroups_rpc [req-a669a1b1-6477-4c3f-907d-271790ad95e4 osstackcanary admin-monitoring - - -] Security group member updated ['85904753-5054-4af3-b8fc-9ad63fe4d765']
Feb 19 05:09:01 cloudvirt1024 CRON[9295]: pam_unix(cron:session): session opened for user prometheus by (uid=0)
Feb 19 05:09:01 cloudvirt1024 CRON[9296]: (prometheus) CMD (/usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom)
Feb 19 05:09:01 cloudvirt1024 systemd[1]: Started Generate prometheus node metrics for cloudvirt ceph network usage.
Feb 19 05:09:01 cloudvirt1024 prometheus-cloudvirt-ceph-network[9298]: [prometheus-cloudvirt-ceph-network] INFO: metrics have been dumped to /var/lib/prometheus/node.d/cloudvirt_ceph_network.prom
Feb 19 05:09:01 cloudvirt1024 systemd[1]: prometheus-node-cloudvirt-ceph-network.service: Succeeded.
Feb 19 05:09:01 cloudvirt1024 CRON[9295]: pam_unix(cron:session): session closed for user prometheus
Feb 19 05:09:03 cloudvirt1024 sudo[9318]: nagios : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/lib/nagios/plugins/check_raid megacli
Feb 19 05:09:03 cloudvirt1024 sudo[9318]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 19 05:09:03 cloudvirt1024 sudo[9318]: pam_unix(sudo:session): session closed for user root
Feb 19 05:09:07 cloudvirt1024 systemd[1]: Starting Collect apt metrics for prometheus-node-exporter...
Feb 19 05:09:09 cloudvirt1024 systemd[1]: prometheus-node-exporter-apt.service: Succeeded.
Feb 19 05:09:09 cloudvirt1024 systemd[1]: Started Collect apt metrics for prometheus-node-exporter.
Feb 19 05:09:11 cloudvirt1024 sshd[9335]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:09:11 cloudvirt1024 sshd[9335]: Connection from 208.80.153.74 port 57910 on 10.64.20.43 port 22
Feb 19 05:09:11 cloudvirt1024 sshd[9335]: Connection closed by 208.80.153.74 port 57910 [preauth]
Feb 19 05:09:16 cloudvirt1024 smartd[1190]: Device: /dev/bus/0 [megaraid_disk_08] [SAT], failed to read SMART Attribute Data
Feb 19 05:09:16 cloudvirt1024 smartd[1190]: Device: /dev/bus/0 [megaraid_disk_08] [SAT], Read SMART Self Test Log Failed
Feb 19 05:09:39 cloudvirt1024 nova-compute[62835]: 2021-02-19 05:09:39.466 62835 WARNING nova.compute.resource_tracker [req-1492a22f-6b0d-44b0-b15f-6a174218312a - - - - -] Instance 98b912c0-8cf7-45f2-8728-942fdcdc803b has been moved to another host cloudvirt1022(cloudvirt1022.eqiad.wmnet). There are allocations remaining against the source host that might need to be removed: {'resources': {'VCPU': 1, 'MEMORY_MB': 2048, 'DISK_GB': 80}}.
Feb 19 05:09:45 cloudvirt1024 sudo[9449]: nagios : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/lib/nagios/plugins/check_puppetrun -w 10800 -c 21600
Feb 19 05:09:45 cloudvirt1024 sudo[9449]: pam_unix(sudo:session): session opened for user root by (uid=0)
Feb 19 05:09:45 cloudvirt1024 sudo[9449]: pam_unix(sudo:session): session closed for user root
Feb 19 05:09:55 cloudvirt1024 sshd[9470]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:09:55 cloudvirt1024 sshd[9470]: Connection from 208.80.153.84 port 41972 on 10.64.20.43 port 22
Feb 19 05:09:55 cloudvirt1024 sshd[9470]: Connection closed by 208.80.153.84 port 41972 [preauth]
Feb 19 05:10:01 cloudvirt1024 CRON[9472]: pam_unix(cron:session): session opened for user prometheus by (uid=0)
Feb 19 05:10:01 cloudvirt1024 CRON[9473]: (prometheus) CMD (/usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom)
Feb 19 05:10:01 cloudvirt1024 systemd[1]: Started Generate prometheus node metrics for cloudvirt ceph network usage.
Feb 19 05:10:01 cloudvirt1024 prometheus-cloudvirt-ceph-network[9475]: [prometheus-cloudvirt-ceph-network] INFO: metrics have been dumped to /var/lib/prometheus/node.d/cloudvirt_ceph_network.prom
Feb 19 05:10:01 cloudvirt1024 systemd[1]: prometheus-node-cloudvirt-ceph-network.service: Succeeded.
Feb 19 05:10:01 cloudvirt1024 CRON[9472]: pam_unix(cron:session): session closed for user prometheus
Feb 19 05:10:07 cloudvirt1024 sshd[9484]: rexec line 19: Deprecated option UsePrivilegeSeparation
Feb 19 05:10:07 cloudvirt1024 sshd[9484]: Connection from 208.80.154.88 port 34826 on 10.64.20.43 port 22
Feb 19 05:10:07 cloudvirt1024 sshd[9484]: Connection closed by 208.80.154.88 port 34826 [preauth]
Feb 19 05:10:15 cloudvirt1024 systemd[1]: Started Intel microcode prometheus metrics exporter.
Feb 19 05:10:15 cloudvirt1024 dbus-daemon[1204]: [system] Activating via systemd: service name='org.freedesktop.timedate1' unit='dbus-org.freedesktop.timedate1.service' requested by ':1.8235085' (uid=109 pid=9498 comm="timedatectl show ")
Feb 19 05:10:15 cloudvirt1024 systemd[1]: Starting Time & Date Service...
Feb 19 05:10:15 cloudvirt1024 systemd[1]: prometheus_intel_microcode.service: Succeeded.
Feb 19 05:10:15 cloudvirt1024 dbus-daemon[1204]: [system] Successfully activated service 'org.freedesktop.timedate1'
Feb 19 05:10:15 cloudvirt1024 systemd[1]: Started Time & Date Service.

Event Timeline

ProdPasteBot changed the title of this paste from untitled to Journal between 2021-02-19 05:00:31 - 2021-02-19 05:10:31.