ATM new ELK7 hosts will get elasticsearch-curator from buster (i.e. 5.2) although that's not compatible with ELK7 and causes cronspam. The buster-wikimedia version should be installed instead
File "/usr/lib/python2.7/dist-packages/curator/utils.py", line 800, in get_client
'Error: {0}'.format(e)
elasticsearch.exceptions.ElasticsearchException: Unable to create client connection to Elasticsearch. Error: Elasticsearch version 7.4.2 incompatible with this version of Curator (5.2.0)| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | colewhite | T272655 Phatality doesn't work with Kibana 7 | |||
| Resolved | herron | T234854 Upgrade ELK Stack to version 7 | |||
| Resolved | herron | T257024 Buster elasticsearch-curator version not compatible with ELK7 |
The logstash::elasticsearch7 nodes (and Pontoon) do not have a curator version available that can run against ES 7.0. A version that can (>=5.7.0) is not available in our apt repository. The logstash7 nodes have elasticsearch-curator 5.8.1, but it is unclear how it got there.
We need elasticsearch-curator >= 5.7.0 in our apt repository.
I found elasticsearch-curator 5.8.1 in the thirdparty/elastic74 component and added it to the thirdparty/elastic710 component. The package is now available for Pontoon and production.
Not closing as I'm not sure if this is the desired final state.
Initially the thinking was to store the appropriate elasticsearch-curator for each ES version in the component. But in practice yeah that's proving to be a bit unclear, esp in cases like this where we've upgraded hosts from one minor version to another and switched components in the process. I think moving curator to thirdparty/curator58 or a similar component would help clear things up and should avoid this type of case in the future.
I think this can be closed: declined.
With the migration to OpenSearch and the future of Curator unclear, I think the path forward will be some solution stemming from T301017.