Page MenuHomePhabricator
Create Task
Maniphest T251466

Upgrade mtail to 3.0.0-rc35
Closed, ResolvedPublic
Actions

Description

A number of issues have held us to ~rc5. We should try out ~rc35 recently added to sid.

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

colewhite created this task.Apr 29 2020, 8:54 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 29 2020, 8:54 PM
colewhite claimed this task.Apr 29 2020, 8:54 PM
colewhite triaged this task as Medium priority.
colewhite added subtasks: T225604: log spam from mtail 3.0.0~rc19 on wezen, T230030: Prometheus failing to ingest some mtail samples.
gerritbot added a comment.Apr 29 2020, 8:58 PM

Change 593314 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] aptrepo: add mtail component for controlled mtail upgrade

https://gerrit.wikimedia.org/r/593314

gerritbot added a project: Patch-For-Review.Apr 29 2020, 8:58 PM
gerritbot added a comment.Apr 29 2020, 9:24 PM

Change 593327 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] mtail: add flag to install mtail apt component

https://gerrit.wikimedia.org/r/593327

Stashbot added a comment.Apr 30 2020, 7:25 PM

Mentioned in SAL (#wikimedia-operations) [2020-04-30T19:25:09Z] <shdubsh> test mtail rc35 upgrade on fermium - T251466

Stashbot added a comment.Apr 30 2020, 8:13 PM

Mentioned in SAL (#wikimedia-operations) [2020-04-30T20:13:04Z] <shdubsh> test mtail rc35 upgrade on logstash1007 - T251466

gerritbot added a comment.May 4 2020, 9:42 PM

Change 594316 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] mtail: update varnishrls compatibility with rc35

https://gerrit.wikimedia.org/r/594316

gerritbot added a comment.May 4 2020, 9:51 PM

Change 594320 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] mtail: update mtail testing to python3

https://gerrit.wikimedia.org/r/594320

gerritbot added a comment.May 5 2020, 3:14 PM

Change 594320 merged by Cwhite:
[operations/puppet@production] mtail: update mtail testing to python3

https://gerrit.wikimedia.org/r/594320

gerritbot added a comment.May 5 2020, 4:01 PM

Change 593314 merged by Cwhite:
[operations/puppet@production] aptrepo: add mtail component for controlled mtail upgrade

https://gerrit.wikimedia.org/r/593314

gerritbot added a comment.May 6 2020, 7:42 PM

Change 593327 merged by Cwhite:
[operations/puppet@production] mtail: add flag to install mtail from apt component

https://gerrit.wikimedia.org/r/593327

fgiunchedi moved this task from Inbox to In progress on the observability board.May 11 2020, 1:58 PM
gerritbot added a comment.May 27 2020, 5:11 PM

Change 594316 merged by Cwhite:
[operations/puppet@production] mtail: update varnishrls compatibility with rc35

https://gerrit.wikimedia.org/r/594316

Maintenance_bot removed a project: Patch-For-Review.May 27 2020, 6:11 PM
gerritbot added a comment.May 28 2020, 3:50 PM

Change 599365 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[integration/config@master] puppet-tests: update mtail to 3.0.0~rc35

https://gerrit.wikimedia.org/r/599365

gerritbot added a project: Patch-For-Review.May 28 2020, 3:50 PM
gerritbot added a comment.May 28 2020, 3:56 PM

Change 599368 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] hiera: install mtail 3.0.0~rc35 from component in eqsin

https://gerrit.wikimedia.org/r/599368

gerritbot added a comment.May 28 2020, 6:53 PM

Change 599368 merged by Cwhite:
[operations/puppet@production] hiera: install mtail 3.0.0~rc35 from component in eqsin

https://gerrit.wikimedia.org/r/599368

gerritbot added a comment.May 28 2020, 11:38 PM

Change 599473 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] hiera: install mtail 3.0.0~rc35 from component in uslfo and codfw

https://gerrit.wikimedia.org/r/599473

gerritbot added a comment.May 28 2020, 11:39 PM

Change 599474 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] hiera: install mtail 3.0.0~rc35 from component in esams and eqiad

https://gerrit.wikimedia.org/r/599474

gerritbot added a comment.Jun 1 2020, 2:58 PM

Change 599473 merged by Cwhite:
[operations/puppet@production] hiera: install mtail 3.0.0~rc35 from component in ulsfo and codfw

https://gerrit.wikimedia.org/r/599473

colewhite added a subtask: T254192: mtail rc35 stops incrementing atsmtail counters.Jun 1 2020, 7:59 PM
gerritbot added a comment.Jun 1 2020, 9:24 PM

Change 601440 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] profile: add additional flags support for atsmtail and disable_fsnotify in eqsin

https://gerrit.wikimedia.org/r/601440

gerritbot added a comment.Jun 2 2020, 9:48 PM

Change 601874 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] hiera: install mtail 3.0.0~rc35 from component in ulsfo and codfw

https://gerrit.wikimedia.org/r/601874

colewhite closed subtask T254192: mtail rc35 stops incrementing atsmtail counters as Resolved.Jun 8 2020, 3:15 PM
gerritbot added a comment.Jun 8 2020, 9:02 PM

Change 601874 merged by Cwhite:
[operations/puppet@production] hiera: install mtail 3.0.0~rc35 from component in ulsfo and codfw

https://gerrit.wikimedia.org/r/601874

gerritbot added a comment.Jun 8 2020, 9:17 PM

Change 603614 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] hiera: set mtail disable_fsnotify in codfw and ulsfo

https://gerrit.wikimedia.org/r/603614

gerritbot added a comment.Jun 8 2020, 9:27 PM

Change 603614 merged by Cwhite:
[operations/puppet@production] hiera: set mtail disable_fsnotify in codfw and ulsfo

https://gerrit.wikimedia.org/r/603614

gerritbot added a comment.Jun 8 2020, 9:55 PM

Change 603626 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] hiera: add disable_fsnotify mtail flag to ncredir

https://gerrit.wikimedia.org/r/603626

gerritbot added a comment.Jun 8 2020, 9:57 PM

Change 603626 merged by Cwhite:
[operations/puppet@production] hiera: add disable_fsnotify mtail flag to ncredir

https://gerrit.wikimedia.org/r/603626

gerritbot added a comment.Jun 8 2020, 10:00 PM

Change 603628 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] hiera: move ncredir config to profile

https://gerrit.wikimedia.org/r/603628

gerritbot added a comment.Jun 8 2020, 10:01 PM

Change 603628 merged by Cwhite:
[operations/puppet@production] hiera: move ncredir config to profile

https://gerrit.wikimedia.org/r/603628

gerritbot added a comment.Jun 8 2020, 11:46 PM

Change 603673 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] hiera: add disable_fsnotify flag for mtail in codfw

https://gerrit.wikimedia.org/r/603673

gerritbot added a comment.Jun 8 2020, 11:48 PM

Change 603673 merged by Cwhite:
[operations/puppet@production] hiera: add disable_fsnotify flag for mtail in codfw

https://gerrit.wikimedia.org/r/603673

gerritbot added a comment.Jun 15 2020, 5:46 PM

Change 599474 merged by Cwhite:
[operations/puppet@production] hiera: install mtail 3.0.0~rc35 from component in esams and eqiad

https://gerrit.wikimedia.org/r/599474

gerritbot added a comment.Jun 15 2020, 9:12 PM

Change 605668 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] hiera: set mtail disable_fsnotify in eqiad and esams

https://gerrit.wikimedia.org/r/605668

gerritbot added a comment.Jun 15 2020, 9:13 PM

Change 605668 merged by Cwhite:
[operations/puppet@production] hiera: set mtail disable_fsnotify in eqiad and esams

https://gerrit.wikimedia.org/r/605668

gerritbot added a comment.Jun 15 2020, 9:18 PM

Change 605669 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] hiera: set disable_fsnotify for cache base in eqiad and esams

https://gerrit.wikimedia.org/r/605669

gerritbot added a comment.Jun 15 2020, 9:19 PM

Change 605669 merged by Cwhite:
[operations/puppet@production] hiera: set disable_fsnotify for cache base in eqiad and esams

https://gerrit.wikimedia.org/r/605669

gerritbot added a comment.Jun 15 2020, 10:31 PM

Change 605688 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] set disable_fsnotify for all current mtail usage

https://gerrit.wikimedia.org/r/605688

gerritbot added a comment.Jun 16 2020, 6:57 AM

Change 599365 merged by jenkins-bot:
[integration/config@master] puppet-tests: update mtail to 3.0.0~rc35

https://gerrit.wikimedia.org/r/599365

gerritbot added a comment.Jun 16 2020, 7:07 AM

Change 605813 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] jjb: update for mtail 3.0.0~rc35

https://gerrit.wikimedia.org/r/605813

gerritbot added a comment.Jun 16 2020, 7:25 AM

Change 605813 merged by jenkins-bot:
[integration/config@master] jjb: update for mtail 3.0.0~rc35

https://gerrit.wikimedia.org/r/605813

hashar subscribed.Jun 16 2020, 7:34 AM

The CI job for operations/puppet should be using 3.0.0~rc35 now. I have filed T255534 to have it run in a standalone job/container

gerritbot added a comment.Jun 24 2020, 9:24 PM

Change 605688 merged by Cwhite:
[operations/puppet@production] set disable_fsnotify for all current mtail usage

https://gerrit.wikimedia.org/r/605688

colewhite closed this task as Resolved.Jun 30 2020, 8:27 PM
colewhite closed subtask T225604: log spam from mtail 3.0.0~rc19 on wezen as Resolved.

mtail rc35 is now deployed across the fleet

colewhite closed subtask T230030: Prometheus failing to ingest some mtail samples as Resolved.Jun 30 2020, 8:31 PM
colewhite closed subtask T255776: mtail "syscall spam" / high cpu usage on logstash1023 as Resolved.Jul 7 2020, 7:36 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits