Page MenuHomePhabricator
Paste P11186

rabbitmqctl cluster status cloudcontrol1004
ActivePublic

Authored by JHedden on May 12 2020, 8:14 PM.
cloudcontrol1004:~$ sudo rabbitmqctl list_queues
Timeout: 60.0 seconds ...
Listing queues for vhost / ...
reply_1ffd520dcef54454a290cc50f6a48bc0 0
reply_0a7ca7b76883414c98246e57e7f21314 0
Error: operation list_queues on node rabbit@cloudcontrol1004 timed out. Timeout value used: 60.0. Some queue(s) are unresponsive, use list_unresponsive_queues command.

Event Timeline

2020-05-12 20:16:04.072 [error] emulator Discarding message {'$gen_call',{<0.2146.0>,#Ref<0.2094199940.271056901.235963>},stat} from <0.2146.0> to <0.7921.0> in an old incarnation (2) of this node (1)

 Starting RabbitMQ 3.7.8 on Erlang 21.2.6
 Copyright (C) 2007-2018 Pivotal Software, Inc.
 Licensed under the MPL.  See http://www.rabbitmq.com/
2020-05-12 20:20:57.663 [info] <0.261.0> 
 node           : rabbit@cloudcontrol1003
 home dir       : /var/lib/rabbitmq
 config file(s) : /etc/rabbitmq/rabbitmq.config
 cookie hash    : cnbu5mjmX6EnA/KuQQ1WwQ==
 log(s)         : /var/log/rabbitmq/rabbit@cloudcontrol1003.log
                : /var/log/rabbitmq/rabbit@cloudcontrol1003_upgrade.log
 database dir   : /var/lib/rabbitmq/mnesia/rabbit@cloudcontrol1003
2020-05-12 20:20:58.737 [info] <0.269.0> Memory high watermark set to 25717 MiB (26966397747 bytes) of 64292 MiB (67415994368 bytes) total
2020-05-12 20:20:58.743 [info] <0.271.0> Enabling free disk space monitoring
2020-05-12 20:20:58.743 [info] <0.271.0> Disk free limit set to 50MB
2020-05-12 20:20:58.747 [info] <0.274.0> Limiting to approx 65436 file handles (58890 sockets)
2020-05-12 20:20:58.748 [info] <0.275.0> FHC read buffering:  OFF
2020-05-12 20:20:58.748 [info] <0.275.0> FHC write buffering: ON
2020-05-12 20:20:58.755 [info] <0.261.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
2020-05-12 20:21:28.756 [warning] <0.261.0> Error while waiting for Mnesia tables: {timeout_waiting_for_tables,[rabbit_user,rabbit_user_permission,rabbit_topic_permission,rabbit_vhost,rabbit_durable_route,rabbit_durable_exchange,rabbit_runtime_parameters,rabbit_durable_queue]}
2020-05-12 20:21:28.756 [info] <0.261.0> Waiting for Mnesia tables for 30000 ms, 8 retries left
2020-05-12 20:21:58.757 [warning] <0.261.0> Error while waiting for Mnesia tables: {timeout_waiting_for_tables,[rabbit_user,rabbit_user_permission,rabbit_topic_permission,rabbit_vhost,rabbit_durable_route,rabbit_durable_exchange,rabbit_runtime_parameters,rabbit_durable_queue]}
2020-05-12 20:21:58.757 [info] <0.261.0> Waiting for Mnesia tables for 30000 ms, 7 retries left
2020-05-12 20:22:28.758 [warning] <0.261.0> Error while waiting for Mnesia tables: {timeout_waiting_for_tables,[rabbit_user,rabbit_user_permission,rabbit_topic_permission,rabbit_vhost,rabbit_durable_route,rabbit_durable_exchange,rabbit_runtime_parameters,rabbit_durable_queue]}
2020-05-12 20:22:28.759 [info] <0.261.0> Waiting for Mnesia tables for 30000 ms, 6 retries left