Maniphest T178845

Cassandra cluster name mismatch warnings
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Eevans
	Oct 23 2017, 9:54 PM

Tags

Referenced Files

None

Subscribers

Description

Hosts in the new RESTBase Cassandra cluster are issuing WARN messages for mismatched cluster names (eqiad!=restbase):

https://logstash.wikimedia.org/goto/bcb6f456a9b70f7184d32549102be97d

Cluster name eqiad is legacy cluster, and restbase is the new cluster (running Cassandra 3.11.0). The message would seem to indicate that hosts in the old cluster are attempting to Gossip with hosts in the new cluster.

Upon cursory examination, the lists of seeds in both clusters seems correct, so this may be old state from a previous misconfiguration. More research is needed.

Related Objects

Mentioned In: T181689: New RESTBase Cassandra cluster has legacy tables

Event Timeline

Eevans created this task.Oct 23 2017, 9:54 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 23 2017, 9:54 PM

Eevans triaged this task as Medium priority.Oct 23 2017, 9:55 PM

Eevans added projects: Services (doing), Cassandra.

Mentioned in SAL (#wikimedia-operations) [2017-10-24T15:53:38Z] <urandom> T178845: Rolling Cassandra restart, codfw, row b

Mentioned in SAL (#wikimedia-operations) [2017-10-24T17:18:23Z] <urandom> T178845: Rolling Cassandra restart, eqiad

Restarting the legacy cluster resolved this; Closing issue.

Eevans mentioned this in T181689: New RESTBase Cassandra cluster has legacy tables.Nov 30 2017, 3:24 PM