Page MenuHomePhabricator

Upgrade Logstash to 1.5.3
Closed, ResolvedPublic

Description

Upgrade from logstash 1.4.2 to latest 1.5.x release. Many many upstream changes in the last year. See https://github.com/elastic/logstash/blob/master/CHANGELOG

Related Objects

Event Timeline

bd808 raised the priority of this task from to Medium.
bd808 updated the task description. (Show Details)
bd808 added a subscriber: bd808.

Change 226991 had a related patch set uploaded (by BryanDavis):
[WIP] Update configuration for logstash 1.5.3

https://gerrit.wikimedia.org/r/226991

Change 227375 had a related patch set uploaded (by BryanDavis):
Initialise repo

https://gerrit.wikimedia.org/r/227375

Change 229073 had a related patch set uploaded (by BryanDavis):
Add logstash-filter-prune 0.1.5

https://gerrit.wikimedia.org/r/229073

Change 229073 merged by BryanDavis:
Add logstash-filter-prune 0.1.5

https://gerrit.wikimedia.org/r/229073

bd808 renamed this task from Upgrade Logstash to 1.5.0 to Upgrade Logstash to 1.5.3.Aug 4 2015, 6:23 PM
bd808 updated the task description. (Show Details)
bd808 set Security to None.

These code changes are now deployed in the beta cluster. Assuming we don't see anything horrible happen there in the new 2-3 days this will be ready to go to production next week.

Pinged SRE about support needs during production deploy window. Tentatively planning to deploy on 2015-08-11. (edited to correct off-by-one error in month)

Deployment plan:

  1. Disable puppet on logstash100[1-3]:
    1. sudo puppet agent --disable "bd808 upgrading logstash to 1.5.3"
  2. Merge needed ops/puppet patches (needs SRE help):
    1. https://gerrit.wikimedia.org/r/#/c/226991/
    2. https://gerrit.wikimedia.org/r/#/c/227175/
    3. https://gerrit.wikimedia.org/r/#/c/230250/
  3. Force puppet run on tin to setup new logstash/plugins Trebuchet repo (needs SRE help)
  4. Tag initial logstash/plugins state
    1. cd /srv/deployment/logstash/plugins
    2. git deploy start
    3. git deploy sync
  5. Update each of logstash100[1-3] sequentially:
    1. sudo service logstash stop
    2. sudo apt-get remove logstash-contrib
    3. sudo apt-get install logstash (1.5.3 is in apt.wm.o now)
    4. sudo puppet agent --enable
    5. sudo puppet agent --test --verbose
    6. Verify:
      1. /etc/logstash/* config updated
      2. /srv/deployment/logstash/plugins present
      3. logstash started
      4. no errors in /var/log/logstash.{err,log}

Change 226991 merged by Filippo Giunchedi:
Update configuration for logstash 1.5.3

https://gerrit.wikimedia.org/r/226991