Page MenuHomePhabricator

Upgrade Logstash to 1.5.3
Closed, ResolvedPublic

Description

Upgrade from logstash 1.4.2 to latest 1.5.x release. Many many upstream changes in the last year. See https://github.com/elastic/logstash/blob/master/CHANGELOG

Related Objects

Event Timeline

bd808 raised the priority of this task from to Normal.
bd808 updated the task description. (Show Details)
bd808 added a subscriber: bd808.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 19 2015, 9:54 PM

Change 226991 had a related patch set uploaded (by BryanDavis):
[WIP] Update configuration for logstash 1.5.3

https://gerrit.wikimedia.org/r/226991

Change 227375 had a related patch set uploaded (by BryanDavis):
Initialise repo

https://gerrit.wikimedia.org/r/227375

Change 227375 merged by BryanDavis:
Initialise repo

https://gerrit.wikimedia.org/r/227375

Change 229073 had a related patch set uploaded (by BryanDavis):
Add logstash-filter-prune 0.1.5

https://gerrit.wikimedia.org/r/229073

Change 229073 merged by BryanDavis:
Add logstash-filter-prune 0.1.5

https://gerrit.wikimedia.org/r/229073

bd808 renamed this task from Upgrade Logstash to 1.5.0 to Upgrade Logstash to 1.5.3.Aug 4 2015, 6:23 PM
bd808 updated the task description. (Show Details)
bd808 set Security to None.

These code changes are now deployed in the beta cluster. Assuming we don't see anything horrible happen there in the new 2-3 days this will be ready to go to production next week.

bd808 added a comment.EditedAug 5 2015, 4:53 AM

Pinged Operations about support needs during production deploy window. Tentatively planning to deploy on 2015-08-11. (edited to correct off-by-one error in month)

bd808 moved this task from To Do to In Dev/Progress on the User-bd808 board.Aug 5 2015, 4:02 PM

Deployment plan:

  1. Disable puppet on logstash100[1-3]:
    1. sudo puppet agent --disable "bd808 upgrading logstash to 1.5.3"
  2. Merge needed ops/puppet patches (needs Operations help):
    1. https://gerrit.wikimedia.org/r/#/c/226991/
    2. https://gerrit.wikimedia.org/r/#/c/227175/
    3. https://gerrit.wikimedia.org/r/#/c/230250/
  3. Force puppet run on tin to setup new logstash/plugins Trebuchet repo (needs Operations help)
  4. Tag initial logstash/plugins state
    1. cd /srv/deployment/logstash/plugins
    2. git deploy start
    3. git deploy sync
  5. Update each of logstash100[1-3] sequentially:
    1. sudo service logstash stop
    2. sudo apt-get remove logstash-contrib
    3. sudo apt-get install logstash (1.5.3 is in apt.wm.o now)
    4. sudo puppet agent --enable
    5. sudo puppet agent --test --verbose
    6. Verify:
      1. /etc/logstash/* config updated
      2. /srv/deployment/logstash/plugins present
      3. logstash started
      4. no errors in /var/log/logstash.{err,log}

Change 226991 merged by Filippo Giunchedi:
Update configuration for logstash 1.5.3

https://gerrit.wikimedia.org/r/226991

bd808 closed this task as Resolved.Aug 11 2015, 4:35 PM
bd808 moved this task from In Dev/Progress to Done on the User-bd808 board.Aug 11 2015, 6:07 PM
bd808 moved this task from Done to Archive on the User-bd808 board.Aug 20 2015, 8:23 PM