Page MenuHomePhabricator

Rollout prometheus-node-exporter 0.14 in labs
Closed, ResolvedPublic

Description

We've got prometheus-node-exporter 0.14.0~git20170523-0 deployed in production that fixes a number of issues (lvs ipv6, qdisc collector, etc). It'd be nice to have it upgraded in labs / tools too, similarly as what was done in T152580: rollout prometheus-node-exporter 0.13

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 30 2017, 2:19 PM
chasemp triaged this task as Medium priority.Jun 1 2017, 12:43 PM
chasemp added a subscriber: chasemp.

sounds good to me, let us know when you hit the Tools road block with aptly and one of us can untangle (i.e. this)

thanks! I think the version in aptly has been removed so we should be set for tools too, what's the best way I can run a command on all labs + tools ?

chasemp added a subscriber: Andrew.Jun 2 2017, 1:47 PM

thanks! I think the version in aptly has been removed so we should be set for tools too, what's the best way I can run a command on all labs + tools ?

There is vanilla clush on tools-puppetmaster-02.tools.eqiad.wmflabs you can do something like clush -f 10 -g all 'foo' and for all of labs it's still salt on labcontrol1001 or a labs wide root running something from local atm. If salt seems to be missing too much @Andrew or I can do a root pass (his is better as a few weird edge instances may still only have his).

fgiunchedi moved this task from Backlog to Doing on the User-fgiunchedi board.Jun 22 2017, 9:56 AM

Did this ever happen? Mostly just curious :)

No it didn't happen, though if someone wants to pick it up please do!

fgiunchedi moved this task from Doing to Backlog on the User-fgiunchedi board.Jul 6 2017, 2:21 PM
fgiunchedi moved this task from Backlog to Doing on the User-fgiunchedi board.

I'll take care of this since we'll need some new collectors from node-exporter 0.14 as part of T177196

Mentioned in SAL (#wikimedia-operations) [2017-10-05T13:15:02Z] <godog> test upgrade node-exporter in thumbor/swift/services/puppet3-diffs/monitoring projects - T166561

fgiunchedi closed this task as Resolved.Oct 5 2017, 3:33 PM

All done! I've ran the upgrade with cumin, using the command below (see also
https://wikitech.wikimedia.org/wiki/Cumin#Upgrade_Debian_packages)

cumin -b 40 'A:all' 'DEBIAN_FRONTEND=noninteractive apt-get -q -y --assume-no -o DPkg::Options::="--force-confdef" -o DPkg::Options::="--force-confold" install prometheus-node-exporter