Page MenuHomePhabricator
Create Task
Maniphest T102008

Import Elasticsearch 1.6.0 deb into wmf apt
Closed, ResolvedPublic
Actions

Description

Please import Elasticsearch 1.6.0 into WMF's apt repository as a step in the direction of us upgrading the production cluster to that version. It was just released yesterday but has a number of features that we're excited about. If all goes super well we'd like to upgrade beta this week and let beta run that version for a week or two before doing production.

  • Stakeholders: Discovery-ARCHIVED, SRE, Security
  • Benefit: Helps us upgrade the production cluster to the latest version of Elasticsearch, allowing much faster restarts after issues and letting us disable dynamic scripting
  • Estimate: To be added

Details

SubjectRepoBranchLines +/-
install_server: switch to elasticsearch 1.6operations/puppetproduction+1 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

• Manybubbles created this task.Jun 10 2015, 3:56 PM
• Manybubbles raised the priority of this task from to Medium.
• Manybubbles updated the task description. (Show Details)
• Manybubbles added projects: Discovery-Search (Current work), acl*sre-team.
• Manybubbles moved this task to Incoming on the Discovery-Search (Current work) board.
• Manybubbles subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 10 2015, 3:56 PM
• Manybubbles added a parent task: T101530: Upgrade Cirrus to Elasticsearch 1.6.0.Jun 10 2015, 3:57 PM
• Manybubbles mentioned this in T101530: Upgrade Cirrus to Elasticsearch 1.6.0.
• Deskana subscribed.Jun 11 2015, 4:51 PM

@Manybubbles Is this an ask for SRE?

• Deskana updated the task description. (Show Details)Jun 11 2015, 4:54 PM
• Deskana set Security to None.
• Manybubbles added subscribers: coren, Joe.Jun 11 2015, 4:57 PM

@Joe, @coren, etc. Is this something an opsen can grab this week? I believe it amounts to a very small amount of work.

• ksmith assigned this task to • Manybubbles.Jun 11 2015, 4:58 PM
• ksmith subscribed.

Assigning to Nik temporarily. As soon as he finds an owner in ops, this can be assigned to that person, and the Discovery sprint project can be removed.

• ksmith moved this task from Incoming to not in use - please delete on the Discovery-Search (Current work) board.Jun 11 2015, 4:59 PM
• Manybubbles added a comment.Jun 23 2015, 8:32 PM

Poking any opsen - can import?

fgiunchedi subscribed.Jun 23 2015, 8:44 PM

@Manybubbles, we can import 1.6 no problem, how long do you think it'll take to validate it? I'm asking because we'll be replacing 1.3 with 1.6 in the repo, thus e.g. new/reimaged machines will get 1.6 not 1.3. How did the beta testing go?

• Manybubbles added a comment.Jun 23 2015, 8:45 PM
In T102008#1394166, @fgiunchedi wrote:

@Manybubbles, we can import 1.6 no problem, how long do you think it'll take to validate it? I'm asking because we'll be replacing 1.3 with 1.6 in the repo, thus e.g. new/reimaged machines will get 1.6 not 1.3. How did the beta testing go?

Development testing went great. I haven't dropped it into beta yet but could do it with curl.

I suspect it'd be better to have both in the repository with puppet forcing the right version, but that is more work than just importing both.

fgiunchedi added a comment.Jun 23 2015, 8:53 PM
In T102008#1394186, @Manybubbles wrote:
In T102008#1394166, @fgiunchedi wrote:

@Manybubbles, we can import 1.6 no problem, how long do you think it'll take to validate it? I'm asking because we'll be replacing 1.3 with 1.6 in the repo, thus e.g. new/reimaged machines will get 1.6 not 1.3. How did the beta testing go?

Development testing went great. I haven't dropped it into beta yet but could do it with curl.

I suspect it'd be better to have both in the repository with puppet forcing the right version, but that is more work than just importing both.

the apt repository manager (reprepro) we use doesn't support multiple versions in the same "distribution" so I think pulling it in beta for now would be easier, I guess a week or so of beta testing?

• ksmith added a parent task: T103599: Update Production to Elasticsearch 1.6.0.Jul 2 2015, 4:50 PM
• Manybubbles added a comment.Jul 6 2015, 5:02 PM
In T102008#1394212, @fgiunchedi wrote:
In T102008#1394186, @Manybubbles wrote:
In T102008#1394166, @fgiunchedi wrote:

@Manybubbles, we can import 1.6 no problem, how long do you think it'll take to validate it? I'm asking because we'll be replacing 1.3 with 1.6 in the repo, thus e.g. new/reimaged machines will get 1.6 not 1.3. How did the beta testing go?

Development testing went great. I haven't dropped it into beta yet but could do it with curl.

I suspect it'd be better to have both in the repository with puppet forcing the right version, but that is more work than just importing both.

the apt repository manager (reprepro) we use doesn't support multiple versions in the same "distribution" so I think pulling it in beta for now would be easier, I guess a week or so of beta testing?

We've been in beta since Monday and everything is a-ok!

When you are ready to import we will schedule the prod upgrade.

Restricted Application added a subscriber: Matanya. · View Herald TranscriptJul 6 2015, 5:02 PM
gerritbot subscribed.Jul 7 2015, 7:49 AM

Change 223251 had a related patch set uploaded (by Filippo Giunchedi):
install_server: switch to elasticsearch 1.6

https://gerrit.wikimedia.org/r/223251

gerritbot added a project: Patch-For-Review.Jul 7 2015, 7:49 AM
bd808 subscribed.Jul 7 2015, 11:48 PM
gerritbot added a comment.Jul 8 2015, 4:10 PM

Change 223251 merged by Filippo Giunchedi:
install_server: switch to elasticsearch 1.6

https://gerrit.wikimedia.org/r/223251

fgiunchedi mentioned this in rOPUP4650c4931f89: install_server: switch to elasticsearch 1.6.Jul 8 2015, 4:10 PM
fgiunchedi closed this task as Resolved.Jul 8 2015, 4:15 PM

you should be all set!

root@carbon:~# reprepro --noskipold checkupdate
aptmethod 'http' seems to have a obsoleted redirect handling which causes
reprepro to request files multiple times. Work-around activated, but better
only use it for targets not redirecting (or upgrade to apt >= 0.9.4 if
that is the http method from apt)!
Calculating packages to get...
Updates needed for 'jessie-wikimedia|thirdparty|source':
Updates needed for 'jessie-wikimedia|thirdparty|amd64':
Updates needed for 'trusty-wikimedia|thirdparty|source':
Updates needed for 'trusty-wikimedia|thirdparty|amd64':
'elasticsearch': '1.3.9' will be upgraded to '1.6.0' (from 'elasticsearch'):
 files needed: pool/thirdparty/e/elasticsearch/elasticsearch_1.6.0_all.deb
Updates needed for 'precise-wikimedia|thirdparty|source':
Updates needed for 'precise-wikimedia|thirdparty|amd64':
'elasticsearch': '1.3.9' will be upgraded to '1.6.0' (from 'elasticsearch'):
 files needed: pool/thirdparty/e/elasticsearch/elasticsearch_1.6.0_all.deb
Updates needed for 'lucid-wikimedia|thirdparty|source':
Updates needed for 'lucid-wikimedia|thirdparty|amd64':
root@carbon:~# reprepro --noskipold --restrict elasticsearch update
aptmethod 'http' seems to have a obsoleted redirect handling which causes
reprepro to request files multiple times. Work-around activated, but better
only use it for targets not redirecting (or upgrade to apt >= 0.9.4 if
that is the http method from apt)!
Calculating packages to get...
Getting packages...
Installing (and possibly deleting) packages...
Exporting indices...
Deleting files no longer referenced...
root@carbon:~# reprepro list trusty-wikimedia elasticsearch
trusty-wikimedia|thirdparty|amd64: elasticsearch 1.6.0
trusty-wikimedia|thirdparty|i386: elasticsearch 1.3.6
root@carbon:~# reprepro --architecture i386 remove trusty-wikimedia elasticsearch
Exporting indices...
root@carbon:~# reprepro list trusty-wikimedia elasticsearch
trusty-wikimedia|thirdparty|amd64: elasticsearch 1.6.0
root@carbon:~#
• Manybubbles moved this task from not in use - please delete to Needs Reporting on the Discovery-Search (Current work) board.Jul 9 2015, 4:18 PM
• Deskana moved this task from Needs Reporting to Resolved on the Discovery-Search (Current work) board.Sep 9 2015, 2:25 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL