Page MenuHomePhabricator

Jclark-ctr (John Clark)
User

Projects

User does not belong to any projects.

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Friday

  • Clear sailing ahead.

User Details

User Since
Jul 24 2019, 8:11 PM (25 w, 6 d)
Availability
Available
LDAP User
Jclark-ctr
MediaWiki User
Jclark-ctr [ Global Accounts ]

Recent Activity

Tue, Jan 14

Jclark-ctr added a comment to T236327: replace onboard NIC in kafka-jumbo100[1-6].

@Cmjohnson no nic installed or host moved yet. @RobH had helped with 10g interfaces

Tue, Jan 14, 7:08 PM · ops-eqiad, Operations, Analytics, User-Elukey
Jclark-ctr reassigned T225121: (Need By: Sept 30) upgrade msw1-eqiad from EX4200 to EX4300 from Jclark-ctr to Cmjohnson.
Tue, Jan 14, 3:28 PM · netops, Operations, ops-eqiad
Jclark-ctr added a comment to T241506: Degraded RAID on db1100.

Replaced Disk #0

Tue, Jan 14, 2:16 PM · DBA, ops-eqiad, Operations

Fri, Jan 10

Jclark-ctr added a comment to T241506: Degraded RAID on db1100.

slot appears to be 0 as discussed on irc we will change monday

Fri, Jan 10, 1:53 PM · DBA, ops-eqiad, Operations

Thu, Jan 9

Jclark-ctr added a comment to T241506: Degraded RAID on db1100.

@Marostegui Drive has arrives Please PM me on IRC so we can get this swapped

Thu, Jan 9, 9:48 PM · DBA, ops-eqiad, Operations

Wed, Jan 8

Jclark-ctr added a comment to T241882: cloudvirt1016 crash.

Confirmed: Service Request 1009577756 was successfully submitted.

Wed, Jan 8, 9:31 PM · cloud-services-team (Hardware), ops-eqiad, DC-Ops, Operations
Jclark-ctr added a comment to T241882: cloudvirt1016 crash.

Confirmed: Service Request 1009577756 was successfully submitted.

Wed, Jan 8, 9:29 PM · cloud-services-team (Hardware), ops-eqiad, DC-Ops, Operations
Jclark-ctr added a comment to T241506: Degraded RAID on db1100.

Drive was ordered should arrive shortly will update when it arrives

Wed, Jan 8, 9:22 PM · DBA, ops-eqiad, Operations

Fri, Jan 3

Jclark-ctr added a comment to T239597: Hardware asset tag Netbox/DNS mgmt inconsistencies.

Verified host asset tags updated ticket . Corrected labstore1007 in netbox.

Fri, Jan 3, 11:25 PM · Operations, ops-eqiad, DC-Ops
Jclark-ctr updated the task description for T239597: Hardware asset tag Netbox/DNS mgmt inconsistencies.
Fri, Jan 3, 11:24 PM · Operations, ops-eqiad, DC-Ops
Jclark-ctr updated the task description for T239597: Hardware asset tag Netbox/DNS mgmt inconsistencies.
Fri, Jan 3, 11:21 PM · Operations, ops-eqiad, DC-Ops
Jclark-ctr updated the task description for T239597: Hardware asset tag Netbox/DNS mgmt inconsistencies.
Fri, Jan 3, 11:17 PM · Operations, ops-eqiad, DC-Ops

Mon, Dec 23

Jclark-ctr added a comment to T240534: Degraded RAID on db1123.

drive changed

Mon, Dec 23, 2:40 PM · DBA, ops-eqiad, Operations

Dec 20 2019

Jclark-ctr added a comment to T236437: (Need By Dec 20) rack/setup/install mw13[49-84].eqiad.wmnet.

@jijiki small delay had a few tickets become urgent. Being a contractor i will be in week after christmas and will wrap them up at beginning of new year.

Dec 20 2019, 11:59 PM · serviceops, Operations, ops-eqiad
Jclark-ctr closed Unknown Object (Task), a subtask of T221632: Storage capacity upgrade for WDQS, as Resolved.
Dec 20 2019, 11:27 PM · Discovery-Search, Wikidata, Wikidata-Query-Service
Jclark-ctr closed Unknown Object (Task), a subtask of T222098: Replace wdqs1003, as Resolved.
Dec 20 2019, 11:27 PM · Wikidata, Wikidata-Query-Service
Jclark-ctr closed Unknown Object (Task), a subtask of T226704: Setup es4 and es5 replica sets for new read-write external store service, as Resolved.
Dec 20 2019, 10:59 PM · Epic, DBA
Jclark-ctr added a comment to T240534: Degraded RAID on db1123.

@Marostegui disk arrived today message me on irc if available to change

Dec 20 2019, 9:45 PM · DBA, ops-eqiad, Operations

Dec 18 2019

Jclark-ctr added a comment to T240534: Degraded RAID on db1123.

Confirmed: Service Request 1007375142 was successfully submitted

Dec 18 2019, 10:12 PM · DBA, ops-eqiad, Operations
Jclark-ctr added a comment to T220698: Check if a GPU fits in any of the remaining stat or notebook hosts.


@EBernhardson stat1004 and 1007 are 1 u host these will not fit dual-slot card . most on this list are 1u host and have same configuration

Dec 18 2019, 2:28 PM · User-Elukey, Analytics, Operations, ops-eqiad

Dec 17 2019

Jclark-ctr added a comment to T236437: (Need By Dec 20) rack/setup/install mw13[49-84].eqiad.wmnet.

I recommend we use these racks we are limited in space . In row D and would not have the enough sfp to rj45 to accommodate the putting these in 10g racks
D 1: 15 servers
D 3: 3 servers
D 6: 15 servers
D 8: 3 servers

Dec 17 2019, 10:24 PM · serviceops, Operations, ops-eqiad

Dec 12 2019

Jclark-ctr claimed T205507: Decommission analytics100[1,2].
Dec 12 2019, 11:17 PM · Operations, ops-eqiad, decommission, Analytics
Jclark-ctr claimed T208586: Decommission lvs1007-1012.
Dec 12 2019, 11:15 PM · ops-eqiad, decommission, Traffic, Operations
Jclark-ctr claimed T208584: Decommission old eqiad caches.
Dec 12 2019, 11:15 PM · ops-eqiad, decommission, Operations, Traffic
Jclark-ctr claimed T234909: decommission auth1001.
Dec 12 2019, 11:04 PM · ops-eqiad, Operations, DC-Ops, decommission
Jclark-ctr claimed T233071: Decommission db1066.eqiad.wmnet.
Dec 12 2019, 11:03 PM · DC-Ops, ops-eqiad, decommission, Operations
Jclark-ctr claimed T146455: Decommission labsdb1002.
Dec 12 2019, 11:03 PM · hardware-requests, Patch-For-Review, ops-eqiad, Operations
Jclark-ctr updated the task description for T191362: decom promethium/WMF3571.
Dec 12 2019, 10:40 PM · decommission, Operations, DC-Ops, ops-eqiad
Jclark-ctr reassigned T239250: setup/install cescout1001.eqiad.wmnet from Jclark-ctr to RobH.
Dec 12 2019, 8:47 PM · Patch-For-Review, ops-eqiad, Operations
Jclark-ctr updated the task description for T239250: setup/install cescout1001.eqiad.wmnet.
Dec 12 2019, 8:46 PM · Patch-For-Review, ops-eqiad, Operations
Jclark-ctr added a comment to T239250: setup/install cescout1001.eqiad.wmnet.

Labeled Host
swap out the dual 480GB SSDs out for the dual 2TB SATA disks

Dec 12 2019, 8:46 PM · Patch-For-Review, ops-eqiad, Operations
Jclark-ctr closed T240545: Circuit down between cr1-eqiad and cr1-codfw as Resolved.
Dec 12 2019, 4:08 PM · ops-eqiad, netops, Operations
Jclark-ctr added a comment to T240545: Circuit down between cr1-eqiad and cr1-codfw.

Replaced failed Fiber

Dec 12 2019, 4:08 PM · ops-eqiad, netops, Operations
Jclark-ctr claimed T240545: Circuit down between cr1-eqiad and cr1-codfw.
Dec 12 2019, 1:39 PM · ops-eqiad, netops, Operations

Dec 10 2019

Jclark-ctr added a comment to T220698: Check if a GPU fits in any of the remaining stat or notebook hosts.

@RobH
Inspected stat1004 with @elukey this morning{F31467984} . 2 available slots. will fit 1 full height and 1 half height .

Dec 10 2019, 11:27 PM · User-Elukey, Analytics, Operations, ops-eqiad
Jclark-ctr updated the task description for T236327: replace onboard NIC in kafka-jumbo100[1-6].
Dec 10 2019, 10:57 PM · ops-eqiad, Operations, Analytics, User-Elukey
Jclark-ctr added a comment to T236327: replace onboard NIC in kafka-jumbo100[1-6].

@RobH
Server New Rack Switchport
kafka-jumbo1001 a4 39
kafka-jumbo1003 b2 35
kafka-jumbo1006 d7 36

Dec 10 2019, 10:44 PM · ops-eqiad, Operations, Analytics, User-Elukey
Jclark-ctr added a comment to T220853: VMs on cloudvirt1015 crashing - bad mainboard/memory.

sent TSR report after running onboard diagnostics that had faults for memory and psu1 & psu2 . TSR report showed no errors running more test..

Dec 10 2019, 9:39 PM · Operations, ops-eqiad, DC-Ops, User-Zppix, cloud-services-team (Kanban)
Jclark-ctr closed T239217: Degraded RAID on dbstore1003 as Resolved.

Replaced Failed Drive

Dec 10 2019, 2:15 PM · Analytics, ops-eqiad, Operations

Dec 6 2019

Jclark-ctr added a comment to T239957: Degraded RAID on cloudelastic1002.

closing due to duplicate .

Dec 6 2019, 1:44 PM · Discovery-Search (Current work), Discovery, ops-eqiad, Operations

Dec 5 2019

Jclark-ctr added a comment to T230088: cloudelastic1002: SMART/disk error.

replaced failed drive

Dec 5 2019, 10:20 PM · ops-eqiad, DC-Ops, Operations, cloud-services-team (Kanban)
Jclark-ctr updated the task description for T235685: (No Need By Date Provided) rack/setup/install cloudvirt-wdqs100[123].eqiad.wmnet.
Dec 5 2019, 8:29 PM · cloud-services-team (Hardware), ops-eqiad, Operations
Jclark-ctr added a comment to T239365: Degraded RAID on an-worker1089.

Replaced failed drive

Dec 5 2019, 8:02 PM · Analytics, ops-eqiad, Operations
Jclark-ctr added a comment to T230088: cloudelastic1002: SMART/disk error.

@mathew.onip Disk arrived

Dec 5 2019, 6:53 PM · ops-eqiad, DC-Ops, Operations, cloud-services-team (Kanban)
Jclark-ctr added a comment to T236437: (Need By Dec 20) rack/setup/install mw13[49-84].eqiad.wmnet.

@jijiki If no surprises i could have them Racked by Dec 20th

Dec 5 2019, 5:42 PM · serviceops, Operations, ops-eqiad

Dec 4 2019

Jclark-ctr added a comment to T239569: cloudstore1008 crash - Memory error.

Finished replacement of DIMM_B2

Dec 4 2019, 10:45 PM · Operations, ops-eqiad, cloud-services-team (Kanban), Cloud-Services
Jclark-ctr added a comment to T239569: cloudstore1008 crash - Memory error.

Dimm has arrived

Dec 4 2019, 10:09 PM · Operations, ops-eqiad, cloud-services-team (Kanban), Cloud-Services
Jclark-ctr added a comment to T239365: Degraded RAID on an-worker1089.

Disk arrived

Dec 4 2019, 10:01 PM · Analytics, ops-eqiad, Operations

Dec 3 2019

Jclark-ctr added a comment to T239045: analytics1057's BBU is faulty.

@elukey unsure if this is same bbu it is diffrent models. 720xd vs 730xd

Dec 3 2019, 10:54 PM · Analytics, Operations, ops-eqiad
Jclark-ctr added a comment to T230088: cloudelastic1002: SMART/disk error.

Confirmed: Service Request 1005040764 was successfully submitted. ordered replacement drive

Dec 3 2019, 10:35 PM · ops-eqiad, DC-Ops, Operations, cloud-services-team (Kanban)
Jclark-ctr added a comment to T220853: VMs on cloudvirt1015 crashing - bad mainboard/memory.

resent tsr report again waiting on dell

Dec 3 2019, 10:20 PM · Operations, ops-eqiad, DC-Ops, User-Zppix, cloud-services-team (Kanban)
Jclark-ctr closed Unknown Object (Task), a subtask of T238652: Hardware request for Postgres database for censorship monitoring scripts, as Resolved.
Dec 3 2019, 8:28 PM · hardware-requests, Operations
Jclark-ctr added a comment to T239569: cloudstore1008 crash - Memory error.

Confirmed: Service Request 1004932600 was successfully submitted.

Dec 3 2019, 1:40 AM · Operations, ops-eqiad, cloud-services-team (Kanban), Cloud-Services
Jclark-ctr closed Unknown Object (Task), a subtask of T220700: Upgrade kafka-jumbo100[1-6] to 10G NICs (if possible), as Resolved.
Dec 3 2019, 1:08 AM · ops-eqiad, hardware-requests, Operations, Analytics, User-Elukey
Jclark-ctr added a comment to T236327: replace onboard NIC in kafka-jumbo100[1-6].

@elukey Received cards please message me on irc and we can start scheduling replacement

Dec 3 2019, 12:48 AM · ops-eqiad, Operations, Analytics, User-Elukey

Dec 2 2019

Jclark-ctr added a comment to T239217: Degraded RAID on dbstore1003.

@Marostegui just received drive from warehouse. Can the drive be swapped now? can you confirm slot?

Dec 2 2019, 9:29 PM · Analytics, ops-eqiad, Operations

Nov 28 2019

Jclark-ctr added a comment to T239365: Degraded RAID on an-worker1089.

Confirmed: Service Request 1004524493 was successfully submitted.

Nov 28 2019, 12:32 AM · Analytics, ops-eqiad, Operations
Jclark-ctr updated the task description for T231967: Decommission dbproxy1005.eqiad.wmnet.
Nov 28 2019, 12:12 AM · DC-Ops, ops-eqiad, decommission, Operations
Jclark-ctr updated the task description for T231892: Decommission db1073.eqiad.wmnet.
Nov 28 2019, 12:12 AM · Patch-For-Review, DC-Ops, ops-eqiad, decommission, Operations
Jclark-ctr updated the task description for T228956: decommission db1072.eqiad.wmnet.
Nov 28 2019, 12:11 AM · Patch-For-Review, DC-Ops, ops-eqiad, decommission, Operations
Jclark-ctr updated the task description for T227166: decommission db1069.
Nov 28 2019, 12:10 AM · ops-eqiad, Operations, DC-Ops, decommission
Jclark-ctr updated the task description for T226517: Decommission old Kafka analytics brokers: kafka1012,kafka1013,kafka1014,kafka1020,kafka1022,kafka1023.
Nov 28 2019, 12:09 AM · ops-eqiad, DC-Ops, Analytics, decommission, Operations
Jclark-ctr updated the task description for T211826: decommission oxygen.eqiad.wmnet.
Nov 28 2019, 12:08 AM · Operations, ops-eqiad, DC-Ops, decommission
Jclark-ctr updated the task description for T209642: Remove labnodepool1001.eqiad.wmnet.
Nov 28 2019, 12:07 AM · DC-Ops, ops-eqiad, decommission, Operations
Jclark-ctr updated the task description for T209181: Decommission rdb1001, rdb1002, rdb1003, rdb1004, rdb1007, rdb1008.
Nov 28 2019, 12:06 AM · ops-eqiad, decommission, User-jijiki, Operations
Jclark-ctr updated the task description for T203861: decom radium.
Nov 28 2019, 12:05 AM · Patch-For-Review, ops-eqiad, decommission, Operations
Jclark-ctr updated the task description for T191357: decom silver/WMF3434.
Nov 28 2019, 12:04 AM · decommission, Operations, DC-Ops, ops-eqiad
Jclark-ctr updated the task description for T187456: Decommission labstore100[123] and their disk shelves.
Nov 28 2019, 12:03 AM · cloud-services-team (Hardware), decommission, Data-Services, Operations, DC-Ops, ops-eqiad

Nov 25 2019

Jclark-ctr reassigned T239098: Duplicate cable label in cr1-eqiad/cr2-eqiad from Jclark-ctr to ayounsi.

@ayounsi Can you update routers to reflect . Thanks!!

Nov 25 2019, 10:29 PM · DC-Ops, Operations, ops-eqiad
Jclark-ctr added a comment to T239098: Duplicate cable label in cr1-eqiad/cr2-eqiad.

cr2-eqiad:xe-3/3/3 cable incorrect in netbox . correct is 2649

Nov 25 2019, 10:28 PM · DC-Ops, Operations, ops-eqiad
Jclark-ctr added a comment to T239045: analytics1057's BBU is faulty.

@elukey No spare bbu around

Nov 25 2019, 10:08 PM · Analytics, Operations, ops-eqiad
Jclark-ctr updated the task description for T239139: rack/setup/install frdb1003.frack.eqiad.wmnet.
Nov 25 2019, 9:40 PM · Operations, fundraising-tech-ops, ops-eqiad
Jclark-ctr added a comment to T239139: rack/setup/install frdb1003.frack.eqiad.wmnet.

Racked Server

Nov 25 2019, 9:37 PM · Operations, fundraising-tech-ops, ops-eqiad
Jclark-ctr updated the task description for T239139: rack/setup/install frdb1003.frack.eqiad.wmnet.
Nov 25 2019, 9:18 PM · Operations, fundraising-tech-ops, ops-eqiad

Nov 20 2019

Jclark-ctr added a comment to T220853: VMs on cloudvirt1015 crashing - bad mainboard/memory.

followed up with dell regarding results spoke with Madhusudan.Rao@dell.com on phone. he will follow up with Kapeel Pawaskar

Nov 20 2019, 1:55 PM · Operations, ops-eqiad, DC-Ops, User-Zppix, cloud-services-team (Kanban)

Nov 19 2019

Jclark-ctr added a comment to T230575: Degraded RAID on cloudvirt1018.

Replaced Failed drive

Nov 19 2019, 10:19 PM · ops-eqiad, Operations
Jclark-ctr added a comment to T236331: Degraded RAID on cloudvirt1018.

Replaced Failed Drive

Nov 19 2019, 10:16 PM · ops-eqiad, Operations

Nov 18 2019

Jclark-ctr reassigned T237438: (Need By 8/15/19) rack/setup/install ms-be105[7-9].eqiad.wmnet from Jclark-ctr to RobH.

@Cmjohnson entered mgmt password for ms-be1057 again

Nov 18 2019, 9:55 PM · ops-eqiad, User-fgiunchedi, Operations

Nov 13 2019

Jclark-ctr added a comment to T236601: Degraded RAID on elastic1039.

@Gehel replaced drive

Nov 13 2019, 4:54 PM · Discovery-Search (Current work), ops-eqiad, Operations
Jclark-ctr closed T227542: b7-eqiad pdu refresh (Tuesday 11/5 @12pm UTC), a subtask of T226778: Install new PDUs in rows A/B (Top level tracking task), as Resolved.
Nov 13 2019, 4:40 PM · DC-Ops, Operations, ops-eqiad
Jclark-ctr closed T227542: b7-eqiad pdu refresh (Tuesday 11/5 @12pm UTC) as Resolved.

confirmed link and errors cleared from icinga

Nov 13 2019, 4:40 PM · DC-Ops, Operations, ops-eqiad
Jclark-ctr updated the task description for T227542: b7-eqiad pdu refresh (Tuesday 11/5 @12pm UTC).
Nov 13 2019, 4:39 PM · DC-Ops, Operations, ops-eqiad
Jclark-ctr closed Unknown Object (Task), a subtask of T228606: Degraded RAID on elastic1046, as Resolved.
Nov 13 2019, 4:02 PM · Patch-For-Review, Discovery-Search (Current work), ops-eqiad, Operations
Jclark-ctr closed T237133: analytics1062 lost one of its power supplies as Resolved.

alert cleared no errors in icinga

Nov 13 2019, 3:54 PM · Analytics, Operations, Analytics-Cluster, ops-eqiad

Nov 12 2019

Jclark-ctr added a comment to T220853: VMs on cloudvirt1015 crashing - bad mainboard/memory.

Verified performance mode in bios . loaded stress test multiple errors on start up sent errors to dell Requested Support Engineer for replacement

Nov 12 2019, 8:09 PM · Operations, ops-eqiad, DC-Ops, User-Zppix, cloud-services-team (Kanban)
Jclark-ctr added a comment to T220853: VMs on cloudvirt1015 crashing - bad mainboard/memory.

@Andrew want to confirm this box is not in use right now. Need to perform additional test for dell

Nov 12 2019, 7:04 PM · Operations, ops-eqiad, DC-Ops, User-Zppix, cloud-services-team (Kanban)

Nov 7 2019

Jclark-ctr closed T237133: analytics1062 lost one of its power supplies as Resolved.
Nov 7 2019, 10:15 PM · Analytics, Operations, Analytics-Cluster, ops-eqiad
Jclark-ctr added a comment to T237133: analytics1062 lost one of its power supplies.

Received new psu and replaced.

Nov 7 2019, 10:14 PM · Analytics, Operations, Analytics-Cluster, ops-eqiad
Jclark-ctr added a comment to T237438: (Need By 8/15/19) rack/setup/install ms-be105[7-9].eqiad.wmnet.

@RobH Finished configuring bios and mgmt

Nov 7 2019, 12:26 AM · ops-eqiad, User-fgiunchedi, Operations

Nov 6 2019

Jclark-ctr updated the task description for T237438: (Need By 8/15/19) rack/setup/install ms-be105[7-9].eqiad.wmnet.
Nov 6 2019, 10:41 PM · ops-eqiad, User-fgiunchedi, Operations
Jclark-ctr added a comment to T237567: mw1247: IPMI Sensor Status UNKNOWN internal IPMI error.

preformed flea power drain

Nov 6 2019, 9:53 PM · ops-eqiad, Operations
Jclark-ctr added a comment to T236739: frdb1001 has suffered a raid event resulting in /dev/sda going read only.

Replaced Raid controller and bbu

Nov 6 2019, 9:49 PM · fundraising-tech-ops, Fundraising-Backlog, DC-Ops
Jclark-ctr added a comment to T237438: (Need By 8/15/19) rack/setup/install ms-be105[7-9].eqiad.wmnet.

@fgiunchedi The host was put into rack prior to this ticket being made i can move to row D today

Nov 6 2019, 1:17 PM · ops-eqiad, User-fgiunchedi, Operations

Nov 5 2019

Jclark-ctr added a comment to T237438: (Need By 8/15/19) rack/setup/install ms-be105[7-9].eqiad.wmnet.

host racked and cabled.

Nov 5 2019, 11:36 PM · ops-eqiad, User-fgiunchedi, Operations
Jclark-ctr updated the task description for T237438: (Need By 8/15/19) rack/setup/install ms-be105[7-9].eqiad.wmnet.
Nov 5 2019, 11:35 PM · ops-eqiad, User-fgiunchedi, Operations
Jclark-ctr updated the task description for T237438: (Need By 8/15/19) rack/setup/install ms-be105[7-9].eqiad.wmnet.
Nov 5 2019, 11:35 PM · ops-eqiad, User-fgiunchedi, Operations
Jclark-ctr added a comment to T224188: rack/setup/install (3) new osd ceph nodes.

@JHedden . Found host did not have ip address in it. Reentered address and mgnt password

Nov 5 2019, 10:15 PM · Operations, cloud-services-team (Kanban), Cloud-Services
Jclark-ctr reassigned T227542: b7-eqiad pdu refresh (Tuesday 11/5 @12pm UTC) from Cmjohnson to RobH.

Connect to port > 15

Nov 5 2019, 1:24 PM · DC-Ops, Operations, ops-eqiad
Jclark-ctr updated the task description for T227542: b7-eqiad pdu refresh (Tuesday 11/5 @12pm UTC).
Nov 5 2019, 1:23 PM · DC-Ops, Operations, ops-eqiad
Jclark-ctr added a comment to T227542: b7-eqiad pdu refresh (Tuesday 11/5 @12pm UTC).

finished pdu refresh

Nov 5 2019, 1:22 PM · DC-Ops, Operations, ops-eqiad