Page MenuHomePhabricator

RobH (Rob Halsell)
Senior Data Center EngineerAdministrator

Projects (20)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Tuesday

  • Clear sailing ahead.

User Details

User Since
Nov 24 2014, 1:43 PM (502 w, 6 d)
Roles
Administrator
Availability
Available
IRC Nick
RobH
LDAP User
RobH
MediaWiki User
RobH [ Global Accounts ]

My GPG Key fingerprint = CB1F C7E7 0FF8 5DB2 6820 9C7E 75ED 14C7 0245 D22A

I am an Senior Data Center Engineer on Wikimedia's Data Center SRE Team.

Please note that private message via phabricator is not my preferred contact means. Please feel free to contact me (robh) directly via irc/freenode, or email my @wikimedia.org email address.

Recent Activity

Fri, Jul 12

RobH triaged T369951: determine cable ID for CRT-008647 as Medium priority.
Fri, Jul 12, 8:58 PM · ops-drmrs
RobH moved T369947: Q1:rack/setup/install frban1002 from Backlog to Racking Tasks on the ops-eqiad board.
Fri, Jul 12, 8:42 PM · fundraising-tech-ops, ops-eqiad, DC-Ops
RobH added a parent task for T369947: Q1:rack/setup/install frban1002: Unknown Object (Task).
Fri, Jul 12, 8:42 PM · fundraising-tech-ops, ops-eqiad, DC-Ops
RobH created T369947: Q1:rack/setup/install frban1002.
Fri, Jul 12, 8:41 PM · fundraising-tech-ops, ops-eqiad, DC-Ops
RobH moved T369942: Q#:rack/setup/install payments200[456] from Backlog to Racking Tasks on the ops-codfw board.
Fri, Jul 12, 8:24 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH added a parent task for T369942: Q#:rack/setup/install payments200[456]: Unknown Object (Task).
Fri, Jul 12, 8:23 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH created T369942: Q#:rack/setup/install payments200[456].
Fri, Jul 12, 8:23 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH updated subscribers of T369940: Q1:rack/setup/install fran1002.
Fri, Jul 12, 7:40 PM · fundraising-tech-ops, ops-eqiad, DC-Ops
RobH moved T369940: Q1:rack/setup/install fran1002 from Backlog to Racking Tasks on the ops-eqiad board.
Fri, Jul 12, 7:40 PM · fundraising-tech-ops, ops-eqiad, DC-Ops
RobH added a parent task for T369940: Q1:rack/setup/install fran1002: Unknown Object (Task).
Fri, Jul 12, 7:40 PM · fundraising-tech-ops, ops-eqiad, DC-Ops
RobH created T369940: Q1:rack/setup/install fran1002.
Fri, Jul 12, 7:39 PM · fundraising-tech-ops, ops-eqiad, DC-Ops
RobH updated subscribers of T369937: Q1:rack/setup/install civi2002, frpig2002, frpm2002.
Fri, Jul 12, 7:18 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH added a parent task for T369937: Q1:rack/setup/install civi2002, frpig2002, frpm2002: Unknown Object (Task).
Fri, Jul 12, 7:18 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH moved T369937: Q1:rack/setup/install civi2002, frpig2002, frpm2002 from Backlog to Racking Tasks on the ops-codfw board.
Fri, Jul 12, 7:18 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH created T369937: Q1:rack/setup/install civi2002, frpig2002, frpm2002.
Fri, Jul 12, 7:17 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH moved T369935: Q1:rack/setup/install frlog2002 from Backlog to Racking Tasks on the ops-codfw board.
Fri, Jul 12, 7:07 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH added a parent task for T369935: Q1:rack/setup/install frlog2002: Unknown Object (Task).
Fri, Jul 12, 7:07 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH created T369935: Q1:rack/setup/install frlog2002.
Fri, Jul 12, 7:06 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH added a parent task for T369931: Q1:rack/setup/install frban2002: Unknown Object (Task).
Fri, Jul 12, 6:53 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH moved T369931: Q1:rack/setup/install frban2002 from Backlog to Racking Tasks on the ops-codfw board.
Fri, Jul 12, 6:52 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH created T369931: Q1:rack/setup/install frban2002.
Fri, Jul 12, 6:52 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH moved T369922: Q1:rack/setup/install frdb1007 from Backlog to Racking Tasks on the ops-eqiad board.
Fri, Jul 12, 5:02 PM · fundraising-tech-ops, ops-eqiad, DC-Ops
RobH added a parent task for T369922: Q1:rack/setup/install frdb1007: Unknown Object (Task).
Fri, Jul 12, 5:01 PM · fundraising-tech-ops, ops-eqiad, DC-Ops
RobH created T369922: Q1:rack/setup/install frdb1007.
Fri, Jul 12, 5:01 PM · fundraising-tech-ops, ops-eqiad, DC-Ops
RobH added a parent task for T369920: Q1:rack/setup/install frdb200[45]: Unknown Object (Task).
Fri, Jul 12, 4:35 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH moved T369920: Q1:rack/setup/install frdb200[45] from Backlog to Racking Tasks on the ops-codfw board.
Fri, Jul 12, 4:35 PM · fundraising-tech-ops, ops-codfw, DC-Ops
RobH created T369920: Q1:rack/setup/install frdb200[45].
Fri, Jul 12, 4:34 PM · fundraising-tech-ops, ops-codfw, DC-Ops

Thu, Jul 11

RobH added a comment to T360356: Request access to servers Dcops group.

Please add 'lshw' because I use it constantly (since i have root) for determining serials of any items installed in the host, and track hw failures.

Thu, Jul 11, 7:35 PM · User-Elukey, SRE, Infrastructure-Foundations

Wed, Jul 10

RobH moved T369672: Q1:rack/setup/install vrts2002 from Backlog to Racking Tasks on the ops-codfw board.
Wed, Jul 10, 8:28 PM · SRE, collaboration-services, ops-codfw, DC-Ops
RobH moved T369743: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304 from Backlog to Racking Tasks on the ops-eqiad board.
Wed, Jul 10, 6:16 PM · SRE, serviceops, ops-eqiad, DC-Ops
RobH created T369744: wikikube-worker1240 to wikikube-worker1304 implementation tracking.
Wed, Jul 10, 4:55 PM · serviceops
RobH created T369743: Q1:rack/setup/install wikikube-worker1240 to wikikube-worker1304.
Wed, Jul 10, 4:48 PM · SRE, serviceops, ops-eqiad, DC-Ops

Tue, Jul 9

RobH moved T369674: Q1:rack/setup/install vrts1003 from Backlog to Racking Tasks on the ops-eqiad board.
Tue, Jul 9, 10:50 PM · SRE, collaboration-services, ops-eqiad, DC-Ops
RobH created T369674: Q1:rack/setup/install vrts1003.
Tue, Jul 9, 10:49 PM · SRE, collaboration-services, ops-eqiad, DC-Ops
RobH created T369672: Q1:rack/setup/install vrts2002.
Tue, Jul 9, 10:39 PM · SRE, collaboration-services, ops-codfw, DC-Ops
RobH moved T369671: Q1:rack/setup/install gerrit1004 from Backlog to Racking Tasks on the ops-eqiad board.
Tue, Jul 9, 10:32 PM · SRE, collaboration-services, ops-eqiad, DC-Ops
RobH added a parent task for T369671: Q1:rack/setup/install gerrit1004: Unknown Object (Task).
Tue, Jul 9, 10:30 PM · SRE, collaboration-services, ops-eqiad, DC-Ops
RobH created T369671: Q1:rack/setup/install gerrit1004.
Tue, Jul 9, 10:29 PM · SRE, collaboration-services, ops-eqiad, DC-Ops
RobH moved T369670: Q1:rack/setup/install gerrit2003 from Backlog to Racking Tasks on the ops-codfw board.
Tue, Jul 9, 10:19 PM · SRE, collaboration-services, ops-codfw, DC-Ops
RobH added a parent task for T369670: Q1:rack/setup/install gerrit2003: Unknown Object (Task).
Tue, Jul 9, 10:19 PM · SRE, collaboration-services, ops-codfw, DC-Ops
RobH created T369670: Q1:rack/setup/install gerrit2003.
Tue, Jul 9, 10:18 PM · SRE, collaboration-services, ops-codfw, DC-Ops
RobH added a parent task for T369661: Q1:rack/setup/install pc1017: Unknown Object (Task).
Tue, Jul 9, 7:26 PM · DBA, SRE, Data-Persistence, ops-eqiad, DC-Ops
RobH moved T369661: Q1:rack/setup/install pc1017 from Backlog to Racking Tasks on the ops-eqiad board.
Tue, Jul 9, 7:25 PM · DBA, SRE, Data-Persistence, ops-eqiad, DC-Ops
RobH created T369661: Q1:rack/setup/install pc1017.
Tue, Jul 9, 7:25 PM · DBA, SRE, Data-Persistence, ops-eqiad, DC-Ops
RobH added a parent task for T369658: Q1:rack/setup/install pc2017: Unknown Object (Task).
Tue, Jul 9, 6:43 PM · DBA, SRE, Data-Persistence, ops-codfw, DC-Ops
RobH moved T369658: Q1:rack/setup/install pc2017 from Backlog to Racking Tasks on the ops-codfw board.
Tue, Jul 9, 6:42 PM · DBA, SRE, Data-Persistence, ops-codfw, DC-Ops
RobH created T369658: Q1:rack/setup/install pc2017.
Tue, Jul 9, 6:42 PM · DBA, SRE, Data-Persistence, ops-codfw, DC-Ops
RobH added a parent task for T369654: Q1:rack/setup/install db22[21-40]: Unknown Object (Task).
Tue, Jul 9, 6:12 PM · DBA, SRE, ops-codfw, Data-Persistence, DC-Ops
RobH moved T369654: Q1:rack/setup/install db22[21-40] from Backlog to Racking Tasks on the ops-codfw board.
Tue, Jul 9, 6:12 PM · DBA, SRE, ops-codfw, Data-Persistence, DC-Ops
RobH created T369654: Q1:rack/setup/install db22[21-40].
Tue, Jul 9, 6:11 PM · DBA, SRE, ops-codfw, Data-Persistence, DC-Ops

Mon, Jul 8

RobH moved T369566: Q1:rack/setup/install frqueue2003, pay-lb2001, pay-lb2002 from Backlog to Racking Tasks on the ops-codfw board.
Mon, Jul 8, 10:32 PM · ops-codfw, fundraising-tech-ops, DC-Ops
RobH added a parent task for T369566: Q1:rack/setup/install frqueue2003, pay-lb2001, pay-lb2002: Unknown Object (Task).
Mon, Jul 8, 10:32 PM · ops-codfw, fundraising-tech-ops, DC-Ops
RobH created T369566: Q1:rack/setup/install frqueue2003, pay-lb2001, pay-lb2002.
Mon, Jul 8, 10:32 PM · ops-codfw, fundraising-tech-ops, DC-Ops
RobH added a parent task for T369565: Q1:rack/setup/install frnetmon1002, pay-lb1001, pay-lb1002: Unknown Object (Task).
Mon, Jul 8, 10:30 PM · ops-eqiad, fundraising-tech-ops, DC-Ops
RobH moved T369565: Q1:rack/setup/install frnetmon1002, pay-lb1001, pay-lb1002 from Backlog to Racking Tasks on the ops-eqiad board.
Mon, Jul 8, 10:30 PM · ops-eqiad, fundraising-tech-ops, DC-Ops
RobH updated the task description for T369565: Q1:rack/setup/install frnetmon1002, pay-lb1001, pay-lb1002.
Mon, Jul 8, 10:30 PM · ops-eqiad, fundraising-tech-ops, DC-Ops
RobH created T369565: Q1:rack/setup/install frnetmon1002, pay-lb1001, pay-lb1002.
Mon, Jul 8, 10:29 PM · ops-eqiad, fundraising-tech-ops, DC-Ops

Thu, Jun 27

RobH edited Description on DC-Ops.
Thu, Jun 27, 8:17 PM

Tue, Jun 25

RobH placed T365763: Q4: install PCIe NVMe SSDs into eqsin text cp50(1[789]|2[01234] up for grabs.
Tue, Jun 25, 4:27 PM · Traffic, ops-eqsin, DC-Ops
RobH updated the task description for T365763: Q4: install PCIe NVMe SSDs into eqsin text cp50(1[789]|2[01234].
Tue, Jun 25, 4:26 PM · Traffic, ops-eqsin, DC-Ops
RobH closed Unknown Object (Task), a subtask of T365763: Q4: install PCIe NVMe SSDs into eqsin text cp50(1[789]|2[01234], as Resolved.
Tue, Jun 25, 4:24 PM · Traffic, ops-eqsin, DC-Ops

Mon, Jun 24

RobH updated the task description for T365763: Q4: install PCIe NVMe SSDs into eqsin text cp50(1[789]|2[01234].
Mon, Jun 24, 10:03 PM · Traffic, ops-eqsin, DC-Ops
RobH updated the task description for T365763: Q4: install PCIe NVMe SSDs into eqsin text cp50(1[789]|2[01234].
Mon, Jun 24, 10:02 PM · Traffic, ops-eqsin, DC-Ops

Tue, Jun 18

RobH added a comment to T357415: Q3:rack/setup/install ml-staging2003.

emailed our rep about the firmware load erorrs:

Tue, Jun 18, 6:26 PM · SRE, Machine-Learning-Team, ops-codfw, DC-Ops

Mon, Jun 17

RobH moved T367820: Q1:rack/setup/install franio100[1-3] from Backlog to Racking Tasks on the ops-eqiad board.
Mon, Jun 17, 9:08 PM · SRE, ops-eqiad, fundraising-tech-ops, DC-Ops
RobH renamed T367820: Q1:rack/setup/install franio100[1-3] from Q1:rack/setup/install franio200[1-3] to Q1:rack/setup/install franio100[1-3].
Mon, Jun 17, 9:08 PM · SRE, ops-eqiad, fundraising-tech-ops, DC-Ops
RobH added a parent task for T367820: Q1:rack/setup/install franio100[1-3]: Unknown Object (Task).
Mon, Jun 17, 9:06 PM · SRE, ops-eqiad, fundraising-tech-ops, DC-Ops
RobH created T367820: Q1:rack/setup/install franio100[1-3].
Mon, Jun 17, 9:06 PM · SRE, ops-eqiad, fundraising-tech-ops, DC-Ops
RobH moved T367820: Q1:rack/setup/install franio100[1-3] from Backlog to Racking Tasks on the ops-codfw board.
Mon, Jun 17, 9:06 PM · SRE, ops-eqiad, fundraising-tech-ops, DC-Ops
RobH moved T367819: Q1:rack/setup/install franio200[1-3] from Backlog to Racking Tasks on the ops-codfw board.
Mon, Jun 17, 8:59 PM · SRE, fundraising-tech-ops, ops-codfw, DC-Ops
RobH added a parent task for T367819: Q1:rack/setup/install franio200[1-3]: Unknown Object (Task).
Mon, Jun 17, 8:59 PM · SRE, fundraising-tech-ops, ops-codfw, DC-Ops
RobH created T367819: Q1:rack/setup/install franio200[1-3].
Mon, Jun 17, 8:58 PM · SRE, fundraising-tech-ops, ops-codfw, DC-Ops
RobH moved T367816: Q1:rack/setup/install fransc2001 from Backlog to Racking Tasks on the ops-codfw board.
Mon, Jun 17, 8:34 PM · SRE, fundraising-tech-ops, ops-codfw, DC-Ops
RobH added a parent task for T367816: Q1:rack/setup/install fransc2001: Unknown Object (Task).
Mon, Jun 17, 8:34 PM · SRE, fundraising-tech-ops, ops-codfw, DC-Ops
RobH created T367816: Q1:rack/setup/install fransc2001.
Mon, Jun 17, 8:33 PM · SRE, fundraising-tech-ops, ops-codfw, DC-Ops
RobH added a parent task for T367814: Q1:rack/setup/install fransc1001: Unknown Object (Task).
Mon, Jun 17, 8:03 PM · SRE, fundraising-tech-ops, ops-eqiad, DC-Ops
RobH moved T367814: Q1:rack/setup/install fransc1001 from Backlog to Racking Tasks on the ops-eqiad board.
Mon, Jun 17, 8:03 PM · SRE, fundraising-tech-ops, ops-eqiad, DC-Ops
RobH created T367814: Q1:rack/setup/install fransc1001.
Mon, Jun 17, 8:03 PM · SRE, fundraising-tech-ops, ops-eqiad, DC-Ops
RobH moved T367804: Q1:rack/setup/install frand200[12] from Backlog to Racking Tasks on the ops-codfw board.
Mon, Jun 17, 6:48 PM · SRE, fundraising-tech-ops, ops-codfw, DC-Ops
RobH added a parent task for T367804: Q1:rack/setup/install frand200[12]: Unknown Object (Task).
Mon, Jun 17, 6:48 PM · SRE, fundraising-tech-ops, ops-codfw, DC-Ops
RobH updated the task description for T367804: Q1:rack/setup/install frand200[12].
Mon, Jun 17, 6:48 PM · SRE, fundraising-tech-ops, ops-codfw, DC-Ops
RobH created T367804: Q1:rack/setup/install frand200[12].
Mon, Jun 17, 6:47 PM · SRE, fundraising-tech-ops, ops-codfw, DC-Ops
RobH moved T367801: Q1:rack/setup/install fransw1001.frack.eqiad.wmnet from Backlog to Racking Tasks on the ops-eqiad board.
Mon, Jun 17, 6:33 PM · SRE, fundraising-tech-ops, ops-eqiad, DC-Ops
RobH added a parent task for T367801: Q1:rack/setup/install fransw1001.frack.eqiad.wmnet: Unknown Object (Task).
Mon, Jun 17, 6:33 PM · SRE, fundraising-tech-ops, ops-eqiad, DC-Ops
RobH created T367801: Q1:rack/setup/install fransw1001.frack.eqiad.wmnet.
Mon, Jun 17, 6:33 PM · SRE, fundraising-tech-ops, ops-eqiad, DC-Ops
RobH moved T367800: Q1:rack/setup/install fransw200[1-3].frack.codfw.wmnet from Backlog to Racking Tasks on the ops-codfw board.
Mon, Jun 17, 6:12 PM · SRE, fundraising-tech-ops, ops-codfw, DC-Ops
RobH added a parent task for T367800: Q1:rack/setup/install fransw200[1-3].frack.codfw.wmnet: Unknown Object (Task).
Mon, Jun 17, 6:12 PM · SRE, fundraising-tech-ops, ops-codfw, DC-Ops
RobH created T367800: Q1:rack/setup/install fransw200[1-3].frack.codfw.wmnet.
Mon, Jun 17, 6:11 PM · SRE, fundraising-tech-ops, ops-codfw, DC-Ops

Fri, Jun 14

RobH updated the task description for T365763: Q4: install PCIe NVMe SSDs into eqsin text cp50(1[789]|2[01234].
Fri, Jun 14, 10:25 PM · Traffic, ops-eqsin, DC-Ops
RobH updated the task description for T364891: Q4: install PCIe NVMe SSDs into ulsfo text cp40(3[789]|4[01234].
Fri, Jun 14, 8:10 PM · Traffic, ops-ulsfo, DC-Ops

Jun 12 2024

RobH closed T366863: PowerSupplyFailure as Resolved.

had been slightly unseated over time even though velcroed in place as it was fully online when i checked today, but pushed in cables.

Jun 12 2024, 5:21 PM · SRE, DC-Ops, ops-ulsfo
RobH moved T366863: PowerSupplyFailure from Backlog to Hardware Failure / Repair on the ops-ulsfo board.
Jun 12 2024, 4:15 PM · SRE, DC-Ops, ops-ulsfo

Jun 11 2024

RobH closed T367232: netbox model cleanup and additions June 2024 as Resolved.

@wiki_willy : FYI as this was an outstanding task item from May DC Ops meetings. All supermicro models (including GPU model) are now in netbox. Additionally, All dell models have been consolidated via mass edits. Example: If there was 'R440 - Config A 202101' and 'R440 - Config A 202203' they have all been consolidated into ''R440 - Config A'. Same holds true for all models (R450 - Config A, R750xs - Config I, etc...). I also audited all the existing items on generic entries (example: Dell PowerEdge R750) and found their entries only have custom builds, or found them to need to be moved from generic to Config specific entries (about 30 of them). This is now complete, just CCing you in via phab notifications so you are aware of completion and ready to move forward on all new supermicro orders into netbox.

Jun 11 2024, 9:08 PM · DC-Ops
RobH updated the task description for T357415: Q3:rack/setup/install ml-staging2003.
Jun 11 2024, 9:03 PM · SRE, Machine-Learning-Team, ops-codfw, DC-Ops
RobH updated the task description for T367232: netbox model cleanup and additions June 2024.
Jun 11 2024, 9:01 PM · DC-Ops
RobH updated the task description for T367232: netbox model cleanup and additions June 2024.
Jun 11 2024, 8:56 PM · DC-Ops
RobH updated the task description for T367232: netbox model cleanup and additions June 2024.
Jun 11 2024, 8:36 PM · DC-Ops
RobH added a comment to T367232: netbox model cleanup and additions June 2024.
PowerEdge R730 - 1 custom config 
PowerEdge R650 - 2 custom config 
PowerEdge R660 - 4 custom config 
PowerEdge R630 - predates standard configs
PowerEdge R750 - had 8 devices that should have been 'PowerEdge R750 - ConfigI' so moved them, 1 custom device
PowerEdge R750xs - had 30 devices half split to configI of same model and the other half to ConfigH of same model.
Jun 11 2024, 8:35 PM · DC-Ops
RobH updated the task description for T367232: netbox model cleanup and additions June 2024.
Jun 11 2024, 8:05 PM · DC-Ops
RobH updated the task description for T367232: netbox model cleanup and additions June 2024.
Jun 11 2024, 8:01 PM · DC-Ops