Page MenuHomePhabricator

codfw: Testing Out Sample PDUs
Open, LowPublicRequest

Description

This task is to review the technical specs for the following PDU options:

  • Chatsworth 48port -
  • Enconnex 48port -

And then test out the samples at codfw, to determine if they are better alternatives to our existing ServerTech PDUs.

Thanks,
Willy

Event Timeline

wiki_willy created this task.
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Latest update - looks like we need to order a minimum of 100 of the Enconnex (because it's customized), so let's scrap that one. Some additional details I gathered for the Chatsworth are below:

Model Number - EA-3126-C
Cost - $1650
Master/Expansion - None, but the PDUs can be daisy chained together
Evaluation Period - Typically 2-3 weeks

Thanks,
Willy

Specs for Eaton PDU attached:

No Master/Expansion, but PDUs can be linked together
Sample PDU can be sent in 2-3 weeks

fgiunchedi added a subscriber: fgiunchedi.

Once the PDU are installed please let observability know. At minimum we'd need to test librenms discovery and their SNMP MIB to snmp-exporter for pulling power data into Prometheus

Thanks @fgiunchedi . Now that the holidays are over, I'm re-engaging the vendor on discussions. After all the paperwork and stuff, my guess is we'll probably have the sample PDUs onsite in about a month or so, but we'll be sure to update the Phab task as we make progress. Thanks, Willy

Once the PDU are installed please let observability know. At minimum we'd need to test librenms discovery and their SNMP MIB to snmp-exporter for pulling power data into Prometheus

Updating task with the new single row Chatsworth design. It's not already supported by Librenms, so it looks like we would have to add it in. A few other notes I took from our meeting with the Account Reps - 3yr warranty (can usually send RMA in 2days, then we ship broken PDU back), 31 days to test the sample PDU (tho we can keep it longer if needed), 3phase is color coded, clips hold the power plugs in, switching capability available on other models, can swap controller module, field failure rate is less than .5%, MTBF of 1.7532 million hours

Received the test PDU . I tested both colored cables, the one we bought first (red on image) and the one that was sent to us for testing( blue on image) all fits well with the lock mechanism on the PDU. The lock on the PDU secured the cable very well better then the

I will open a ticket with CY1 to come and connect the new PDU for testing

First issue :

  • The mounting buttons don't align with the mounting bracket in the rack
  • The PDU is using USB Temperature and Humidity Sensor or we have RJ11 Temperature and Humidity Sensor
  • The panel is up side down
  • The PDU can only be linked with the same PDU or a PUD that has a RJ45 port

Change 676098 had a related patch set uploaded (by Papaul; author: Papaul):

[operations/puppet@production] Add test pdu ps-test-d8-codfw

https://gerrit.wikimedia.org/r/676098

Change 676098 merged by Papaul:

[operations/puppet@production] Add test pdu ps-test-d8-codfw

https://gerrit.wikimedia.org/r/676098

The test PDU is on online .its been monitor in Librenms
https://librenms.wikimedia.org/device/203
The only thing left is the setup in icinga

@fgiunchedi ^

Sounds good @Papaul ! So in Icinga we're monitoring each phase to see if it hits 80%/85% of the 30A breaker, and in Prometheus we're collecting most of what we can via snmp (current, voltage, sensors).

@wiki_willy looks like the PDU is (somewhat) supported by LibreNMS so that's good news! (although I'm not seeing sensors data and inlet power, maybe that's expected for the sensors?).
Could we ask the vendor which SNMP OIDs we need to use to get input phase(s) data (volt/amp/etc) and sensors data? We could probably work it out ourselves via the MIB but I think better ask. Thank you !

Thanks for the feedback @fgiunchedi. We plan on setting up a follow-up meeting with the vendor next week to provide them some feedback, so we'll be sure to pass along your questions. Let us know though, if you have any other questions/concerns that pop up in the meantime. Thanks, Willy

Sounds good @Papaul ! So in Icinga we're monitoring each phase to see if it hits 80%/85% of the 30A breaker, and in Prometheus we're collecting most of what we can via snmp (current, voltage, sensors).

@wiki_willy looks like the PDU is (somewhat) supported by LibreNMS so that's good news! (although I'm not seeing sensors data and inlet power, maybe that's expected for the sensors?).
Could we ask the vendor which SNMP OIDs we need to use to get input phase(s) data (volt/amp/etc) and sensors data? We could probably work it out ourselves via the MIB but I think better ask. Thank you !