Page MenuHomePhabricator

Update Wikimedia apt repo to include debs for Elasticsearch & Logstash on jessie
Closed, ResolvedPublic

Description

When provisioning the new jessie based Elasticsearch backends for the production Logstash cluster I had to manually curl http://apt.wikimedia.org/wikimedia/pool/thirdparty/e/elasticsearch/elasticsearch_1.3.6_all.deb and install the deb with dpkg (T97645). We should get our local apt repo updated to include the more modern Elasticsearch debs in the jessie index.

Needed debs:

Event Timeline

bd808 created this task.May 4 2015, 6:01 PM
bd808 claimed this task.
bd808 raised the priority of this task from to Needs Triage.
bd808 updated the task description. (Show Details)
bd808 added subscribers: bd808, Manybubbles, demon, Joe.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 4 2015, 6:01 PM
bd808 removed bd808 as the assignee of this task.May 4 2015, 6:01 PM

Having read the comments on T97645, do you want us to just reuse the elasticsearch_1.3.6_all.deb that you manually installed, or are there better packages elsewhere? I had little luck trying to hunt around on packages.elasticsearch.org.

@ArielGlenn, If we locked the version of Elasticsearch that cirrus used then it'd be fine to just include them all. Right now its just ensure => present.

bd808 added a comment.May 14 2015, 4:08 PM

@ArielGlenn, If we locked the version of Elasticsearch that cirrus used then it'd be fine to just include them all. Right now its just ensure => present.

Pinning the version in Puppet is problematic for graceful cluster upgrades isn't it? That would require you to disable Puppet while upgrading the cluster, upgrade, adjust Puppet and then re-enable.

We would probably want to at least move the version to a hiera variable so that that we weren't forced to upgrade all of the various Elasticsearch clusters controlled with Puppet in lock step.

Today, having the current 1.3.6 that is in out thirdparty pool available on jessie would make me happy. We probably need to think about our future upgrade process though. For the Logstash related clusters in prod and beta I'd love to keep lagging a bit behind Nik's work on the Cirrus clusters to minimize the chance of surprises when bumping Elasticsearch versions. Unstable log aggregation can be worse than no log aggregation.

Dzahn triaged this task as Normal priority.May 26 2015, 9:53 PM
Dzahn added a subscriber: Dzahn.

1.3.6 on jessie?

i already see 1.3.9 on terbium and that is precise

The version in standard jessie in 1.0.2, but there's 1.4.5 in experimental which could easily be backported, the current debs ship binary jars provided by upstream.

1.3.6 was the version I installed on the Jessie boxes for Logstash because that matches the version that is installed on the Precise Logstash hosts. Putting 1.3.9 in apt instead of 1.3.6 would be fine. I can do a cluster update for all of them. We have been holding off on newer versions waiting for 1.6.0 to ship upstream.

bd808 added a comment.May 30 2015, 9:16 PM

The current cluster has been updated to elasticsearch_1.3.9_all.deb now so this is the version we should make the default for jessie. I just downloaded the deb from http://apt.wikimedia.org/wikimedia/pool/thirdparty/e/elasticsearch/elasticsearch_1.3.9_all.deb and installed it with dpkg to do the upgrade.

bd808 renamed this task from Update Wikimedia apt repo to include debs for Elasticsearch on jessie to Update Wikimedia apt repo to include debs for Elasticsearch & Logstash on jessie.Jun 5 2015, 11:30 PM
bd808 updated the task description. (Show Details)
bd808 set Security to None.
bd808 added a comment.Jul 9 2015, 3:31 PM

Elasticsearch 1.6.0 should be the default jessie deb now that it the default for precise/trusty.

Restricted Application added a project: Discovery. · View Herald TranscriptJul 9 2015, 3:31 PM
Restricted Application added a subscriber: Matanya. · View Herald Transcript
MoritzMuehlenhoff closed this task as Resolved.Jul 28 2015, 8:12 AM

elasticsearch-1.7.0 has been imported for jessie-wikimedia and trusty-wikimedia.

bd808 moved this task from Backlog to Archive on the Wikimedia-Logstash board.Jul 28 2015, 4:49 PM

1.7.1 has been released, which fixes a data-loss bug, I'll import that version as well:

"The bug in question (#12487) can, in very rare circumstances involving multiple simultaneous node failures or restarts, cause all copies of a shard to be deleted from the cluster. This bug was introduced in 1.5.0."

https://www.elastic.co/blog/elasticsearch-1-7-1-and-1-6-2-released

elasticsearch-1.7.1 has been imported for jessie-wikimedia and trusty-wikimedia.

Restricted Application added a project: Discovery-Search. · View Herald TranscriptAug 19 2019, 2:33 PM