Page MenuHomePhabricator
Feed Advanced Search

Wed, Jun 19

Cmjohnson added a comment to T224188: rack/setup/install (3) new osd ceph nodes.

@Bstorm @ayounsi I will need very clear instructions on which racks/rows these servers can go in before I physically rack and cable. Once that is figured out please update the task.

Wed, Jun 19, 4:57 PM · ops-eqiad, Operations, cloud-services-team (Kanban), Cloud-Services
Cmjohnson updated the task description for T224188: rack/setup/install (3) new osd ceph nodes.
Wed, Jun 19, 4:56 PM · ops-eqiad, Operations, cloud-services-team (Kanban), Cloud-Services
Cmjohnson updated the task description for T225704: eqiad: rack/setup/install (4) dbproxy systems..
Wed, Jun 19, 4:54 PM · Patch-For-Review, Operations, DBA
Cmjohnson reassigned T225704: eqiad: rack/setup/install (4) dbproxy systems. from Cmjohnson to RobH.

I updated the switch config to private1-d.....both servers are currently off and ready for installs. assigning to @RobH to install

Wed, Jun 19, 4:54 PM · Patch-For-Review, Operations, DBA
Cmjohnson added a comment to T222731: Storage problems with new host db1133.

Dell is sending me a new Raid card, cables and backplane. Sorry, it took so long, I had to call them after they denied my second request.

Wed, Jun 19, 4:40 PM · ops-eqiad, Operations
Cmjohnson added a comment to T218544: ms-be1043 sdk failed.

I thought I had a ticket for this open with HPE but it doesn't look that way. I will take care of it ASAP

Wed, Jun 19, 4:38 PM · observability, Operations-Software-Development, Operations, ops-eqiad
Cmjohnson closed T214283: Memory correctable errors -EDAC- elastic1029 as Resolved.

Closing this for now, let me know if there is another issue. Keep in mind this server is out of warranty

Wed, Jun 19, 4:10 PM · Discovery-Search (Current work), ops-eqiad, Discovery, DC-Ops, Operations
Cmjohnson added a comment to T214283: Memory correctable errors -EDAC- elastic1029.

The DIMM has been reseated and swapped to the opposite sides.

Wed, Jun 19, 3:45 PM · Discovery-Search (Current work), ops-eqiad, Discovery, DC-Ops, Operations

Mon, Jun 17

Cmjohnson closed T196697: rack/setup/add to spares tracking 2 single cpu misc class systems as Resolved.

these have been racked

Mon, Jun 17, 6:48 PM · ops-eqiad, Operations
Cmjohnson closed T225219: eqiad: rack and setup (3) dual CPU servers as Resolved.

servers are set up and have been added to the tracking sheet

Mon, Jun 17, 6:44 PM · Operations, ops-eqiad
Cmjohnson updated the task description for T225219: eqiad: rack and setup (3) dual CPU servers .
Mon, Jun 17, 6:44 PM · Operations, ops-eqiad
Cmjohnson closed T219890: rack/setup 3 new single cpu spare pool systems as Resolved.

servers are ready as spares and in tracking sheet

Mon, Jun 17, 6:43 PM · ops-eqiad, Operations
Cmjohnson added a comment to T225704: eqiad: rack/setup/install (4) dbproxy systems..

@ayounsi I rather not move the servers...I racked them based on the instructions and they're already in racks and setup

Mon, Jun 17, 6:39 PM · Patch-For-Review, Operations, DBA
Cmjohnson updated the task description for T219890: rack/setup 3 new single cpu spare pool systems.
Mon, Jun 17, 6:33 PM · ops-eqiad, Operations
Cmjohnson added a comment to T225704: eqiad: rack/setup/install (4) dbproxy systems..

@Marostegui: do they all go to the cloud vlan? if they do then 1020 and 1021 are in row D...that support-cloud vlan is not available on row D yet. I need Arzhel to copy the vlan over.

Mon, Jun 17, 6:17 PM · Patch-For-Review, Operations, DBA
Cmjohnson moved T225704: eqiad: rack/setup/install (4) dbproxy systems. from Backlog to Racking Tasks on the ops-eqiad board.
Mon, Jun 17, 6:11 PM · Patch-For-Review, Operations, DBA
Cmjohnson updated the task description for T225704: eqiad: rack/setup/install (4) dbproxy systems..
Mon, Jun 17, 6:11 PM · Patch-For-Review, Operations, DBA
Cmjohnson reassigned T225704: eqiad: rack/setup/install (4) dbproxy systems. from Cmjohnson to ayounsi.

Assigning to @ayounsi to add cloud-support1-d-eqiad. Once that is done, the vlan for dbproxy1020 and 1021 will need to be set up. Switch port descriptions are done.

Mon, Jun 17, 6:11 PM · Patch-For-Review, Operations, DBA
Cmjohnson updated the task description for T225704: eqiad: rack/setup/install (4) dbproxy systems..
Mon, Jun 17, 6:04 PM · Patch-For-Review, Operations, DBA

Tue, Jun 11

Cmjohnson moved T223518: ms-be1033 not powering up from Hardware Failure / Troubleshoot to Blocked on the ops-eqiad board.
Tue, Jun 11, 4:16 PM · User-fgiunchedi, Operations, ops-eqiad
Cmjohnson moved T215411: thumbor1004 memory errors from Hardware Failure / Troubleshoot to Stalled on the ops-eqiad board.
Tue, Jun 11, 4:16 PM · User-jijiki, Thumbor, ops-eqiad, serviceops, Operations
Cmjohnson added a comment to T222731: Storage problems with new host db1133.

they declined my ticket...says I didn't isolate the problem well enough.

Tue, Jun 11, 4:15 PM · ops-eqiad, Operations
Cmjohnson closed T222922: wmf7622 wont powercycle (cannot be allocated from spares) as Resolved.

This server accepts all the racadm commands successfully. I verified on-site that these things actually happened

Tue, Jun 11, 4:02 PM · Operations, ops-eqiad
Cmjohnson updated the task description for T222922: wmf7622 wont powercycle (cannot be allocated from spares).
Tue, Jun 11, 4:01 PM · Operations, ops-eqiad
Cmjohnson updated the task description for T222922: wmf7622 wont powercycle (cannot be allocated from spares).
Tue, Jun 11, 4:01 PM · Operations, ops-eqiad
Cmjohnson added a comment to T220853: VMs on cloudvirt1015 crashing - bad mainboard/memory.

@Andrew what parts? There is nothing that suggests that it is CPU on the server side of things. I reseated and moved the DIMM and that error has not returned. It may very well have been poorly seated DIMM. I checked dmesg and do not see any more errors related to memory or CPU. Try putting it back into production and let's see if anything comes back. Unfortunately, I need to demonstrate and prove there is a problem for Dell to do anything and right now I do not have anything to give them.

Tue, Jun 11, 3:57 PM · Operations, ops-eqiad, DC-Ops, User-Zppix, cloud-services-team (Kanban)
Cmjohnson moved T224260: restbase-dev1006 has a broken disk from Hardware Failure / Troubleshoot to Blocked on the ops-eqiad board.
Tue, Jun 11, 3:50 PM · Cassandra, RESTBase, Core Platform Team (Security, stability, performance and scalability (TEC1)), Core Platform Team Backlog (Watching / External), Services (watching), Operations, DC-Ops, ops-eqiad
Cmjohnson closed T223825: Degraded RAID on restbase-dev1006 as Declined.

this is a duplicate task declining

Tue, Jun 11, 3:50 PM · ops-eqiad, Operations
Cmjohnson assigned T223825: Degraded RAID on restbase-dev1006 to RobH.

This server's SSD's are not part of the original build and under HP warranty. They are intel SSDs that I believe came from restbase1001-1003. Assigning to @RobH to order new SSDs.

Tue, Jun 11, 3:49 PM · ops-eqiad, Operations
Cmjohnson moved T222950: cloudvirt1006 - RAID battery failed from Hardware Failure / Troubleshoot to Blocked on the ops-eqiad board.
Tue, Jun 11, 3:46 PM · cloud-services-team, ops-eqiad, Operations
Cmjohnson closed T220880: Degraded RAID on analytics1039 as Resolved.

I found a spare disk and added the disk back, it's now online

Tue, Jun 11, 3:45 PM · ops-eqiad, Operations
Cmjohnson added a comment to T214283: Memory correctable errors -EDAC- elastic1029.

@Gehel you will need to take the server offline for a day so I can reseat the DIMM. The server logs do not indicate any memory errors. If you want to downtime it for Wednesday or Thursday let me know.

Tue, Jun 11, 3:37 PM · Discovery-Search (Current work), ops-eqiad, Discovery, DC-Ops, Operations
Cmjohnson closed T225391: db1077 crashed as Resolved.

@Marostegui that log entry may have been old. The server has both power supplies connected and does not report any current errors. Resolving the task.

Tue, Jun 11, 3:35 PM · Patch-For-Review, ops-eqiad, DBA, Operations

Mon, Jun 10

Cmjohnson reassigned T224260: restbase-dev1006 has a broken disk from Cmjohnson to RobH.

@RobH this disk will need to be ordered outside of the warranty. These servers were shipped without disks, the procurement task states that the disk from RBDEV1001-1003 will be used. They are 800GB Intell SSDS

Mon, Jun 10, 7:08 PM · Cassandra, RESTBase, Core Platform Team (Security, stability, performance and scalability (TEC1)), Core Platform Team Backlog (Watching / External), Services (watching), Operations, DC-Ops, ops-eqiad
Cmjohnson closed T223126: Install new PDUs into b5-eqiad as Resolved.

This has been completed

Mon, Jun 10, 6:59 PM · ops-eqiad, Operations
Cmjohnson closed T224795: Degraded RAID on analytics1029 as Declined.

since this server is out of warranty and @elukey said to skip replacing the disk. If the status changes and needs to be done please re-open task

Mon, Jun 10, 6:54 PM · Analytics, ops-eqiad, Operations
Cmjohnson closed T224805: db1062 (s7 db primary master) disk with predictive failure, a subtask of T208323: Predictive failures on disk S.M.A.R.T. status, as Declined.
Mon, Jun 10, 6:53 PM · Operations, DBA
Cmjohnson closed T224805: db1062 (s7 db primary master) disk with predictive failure as Declined.

declining this for now since it's out of warranty and the disk has not failed

Mon, Jun 10, 6:53 PM · ops-eqiad, DBA, Operations
Cmjohnson reassigned T225391: db1077 crashed from Cmjohnson to Marostegui.

I updated with the service pack and powered on...reassigning to @Marostegui

Mon, Jun 10, 6:48 PM · Patch-For-Review, ops-eqiad, DBA, Operations

Fri, Jun 7

Cmjohnson closed T223518: ms-be1033 not powering up as Resolved.

The motherboard was replaced and the server is back up

Fri, Jun 7, 4:22 PM · User-fgiunchedi, Operations, ops-eqiad

Thu, Jun 6

Cmjohnson added a comment to T224805: db1062 (s7 db primary master) disk with predictive failure.

The server is out of warrant and we will need to order more 600GB disks.

Thu, Jun 6, 3:18 PM · ops-eqiad, DBA, Operations
Cmjohnson moved T225219: eqiad: rack and setup (3) dual CPU servers from Backlog to Racking Tasks on the ops-eqiad board.
Thu, Jun 6, 3:17 PM · Operations, ops-eqiad
Cmjohnson added a parent task for T225219: eqiad: rack and setup (3) dual CPU servers : Unknown Object (Task).
Thu, Jun 6, 2:59 PM · Operations, ops-eqiad
Cmjohnson created T225219: eqiad: rack and setup (3) dual CPU servers .
Thu, Jun 6, 2:59 PM · Operations, ops-eqiad
Cmjohnson added a comment to T223518: ms-be1033 not powering up.

The HP technician will be her June 7 @1000 Ashburn time.

Thu, Jun 6, 2:55 PM · User-fgiunchedi, Operations, ops-eqiad
Cmjohnson added a comment to T222731: Storage problems with new host db1133.

You have successfully submitted request SR991779294.

Thu, Jun 6, 2:52 PM · ops-eqiad, Operations

Wed, Jun 5

Cmjohnson added a comment to T222731: Storage problems with new host db1133.

Update on this server. I have updated all of the f/w including the raid card. I am able to isolate the problem to slot 0 right now. I moved the disks around and they do not report any errors only the slot. I have blown out the raid several times and re-configured but the error keeps coming back. I have reseated the raid card as well.

Wed, Jun 5, 5:22 PM · ops-eqiad, Operations
Cmjohnson closed T225060: db1091 crashed as Resolved.

The bbu has been replaced.

Wed, Jun 5, 5:10 PM · Patch-For-Review, ops-eqiad, DBA, Operations
Cmjohnson added a comment to T225060: db1091 crashed.

Good afternoon! db1091...i do have a spare bbu but that spare has been helpful the last year or so. HP is slow to send out the batteries, they can take days to get because of their slow response time and then having to ship batteries via ground transportation only. If I use it for this server than I am not able to quickly change out the bbu on something that may be more important in the future. The call
10:22 is yours since you have the most BBU issues.

Wed, Jun 5, 2:26 PM · Patch-For-Review, ops-eqiad, DBA, Operations
Cmjohnson moved T223217: Decommission db1064 from Backlog to Decommission on the ops-eqiad board.
Wed, Jun 5, 2:19 PM · Operations, ops-eqiad, decommission

Tue, Jun 4

Cmjohnson updated the task description for T219890: rack/setup 3 new single cpu spare pool systems.
Tue, Jun 4, 4:04 PM · ops-eqiad, Operations

Fri, May 31

Cmjohnson closed T213422: es1019 IPMI and its management interface are unresponsive (again) as Resolved.

@jcrespo the server is back on and I am able to reach the mgmt interface.

Fri, May 31, 6:41 PM · Patch-For-Review, Operations, ops-eqiad
Cmjohnson closed T213422: es1019 IPMI and its management interface are unresponsive (again), a subtask of T167121: Several hosts return "internal IPMI error" in the check_ipmi_temp check, as Resolved.
Fri, May 31, 6:41 PM · Patch-For-Review, Operations, observability
Cmjohnson closed T213422: es1019 IPMI and its management interface are unresponsive (again), a subtask of T193155: IPMI Audit 2018-04, as Resolved.
Fri, May 31, 6:40 PM · Operations
Cmjohnson assigned T207707: contint1001 store docker images on separate partition or disk to greg.

@greg the disks have been added and assigned to you

Fri, May 31, 6:14 PM · serviceops, Release-Engineering-Team (Kanban), Operations, Continuous-Integration-Infrastructure

Thu, May 30

Cmjohnson added a comment to T220853: VMs on cloudvirt1015 crashing - bad mainboard/memory.

Swapped DIMM B3 with DIMM A3 and cleared the log.

Thu, May 30, 5:39 PM · Operations, ops-eqiad, DC-Ops, User-Zppix, cloud-services-team (Kanban)
Cmjohnson added a comment to T221502: db1099 memory issues.

Swapped DIMM A5 with DIMM B5 and cleared the racadm log.

Thu, May 30, 3:42 PM · ops-eqiad, DBA, Operations
Cmjohnson added a comment to T207707: contint1001 store docker images on separate partition or disk.

@greg @RobH I am just plugging these disks into the server correct? nothing else? this will not require downtime afaik.

Thu, May 30, 2:38 PM · serviceops, Release-Engineering-Team (Kanban), Operations, Continuous-Integration-Infrastructure
Cmjohnson closed Unknown Object (Task), a subtask of T207707: contint1001 store docker images on separate partition or disk, as Resolved.
Thu, May 30, 2:36 PM · serviceops, Release-Engineering-Team (Kanban), Operations, Continuous-Integration-Infrastructure

Wed, May 29

Cmjohnson added a comment to T223825: Degraded RAID on restbase-dev1006.

a ticket has been created with HP for a replacement 5338974144

Wed, May 29, 7:06 PM · ops-eqiad, Operations
Cmjohnson added a comment to T223518: ms-be1033 not powering up.

Steps i have taken

Wed, May 29, 7:00 PM · User-fgiunchedi, Operations, ops-eqiad
Cmjohnson assigned T222950: cloudvirt1006 - RAID battery failed to RobH.

this server is out of warranty. @RobH should we order a new battery?

Wed, May 29, 6:37 PM · cloud-services-team, ops-eqiad, Operations
Cmjohnson added a comment to T220880: Degraded RAID on analytics1039.

@elukey I do not have any 4TB disks left over in eqiad. If I understand your comment correctly you are saying it's okay to ignore this for now.

Wed, May 29, 6:36 PM · ops-eqiad, Operations

Tue, May 28

Cmjohnson moved T222620: cp1083 crashed from Backlog to Stalled on the ops-eqiad board.
Tue, May 28, 2:55 PM · Operations, ops-eqiad, Traffic
Cmjohnson moved T213422: es1019 IPMI and its management interface are unresponsive (again) from Backlog to Hardware Failure / Troubleshoot on the ops-eqiad board.
Tue, May 28, 2:54 PM · Patch-For-Review, Operations, ops-eqiad
Cmjohnson moved T214283: Memory correctable errors -EDAC- elastic1029 from Backlog to Hardware Failure / Troubleshoot on the ops-eqiad board.
Tue, May 28, 2:54 PM · Discovery-Search (Current work), ops-eqiad, Discovery, DC-Ops, Operations
Cmjohnson moved T220698: Check if a GPU fits in any of the remaining stat or notebook hosts from Backlog to Not urgent on the ops-eqiad board.
Tue, May 28, 2:54 PM · User-Elukey, Analytics, Operations, ops-eqiad
Cmjohnson moved T220880: Degraded RAID on analytics1039 from Backlog to Hardware Failure / Troubleshoot on the ops-eqiad board.
Tue, May 28, 2:54 PM · ops-eqiad, Operations
Cmjohnson moved T222950: cloudvirt1006 - RAID battery failed from Backlog to Hardware Failure / Troubleshoot on the ops-eqiad board.
Tue, May 28, 2:53 PM · cloud-services-team, ops-eqiad, Operations
Cmjohnson moved T223450: Triage and resolve all outstanding Netbox report errors from Backlog to Not urgent on the ops-eqiad board.
Tue, May 28, 2:53 PM · ops-codfw, ops-eqiad, Operations, Operations-Software-Development, netbox, DC-Ops
Cmjohnson moved T223518: ms-be1033 not powering up from Backlog to Hardware Failure / Troubleshoot on the ops-eqiad board.
Tue, May 28, 2:53 PM · User-fgiunchedi, Operations, ops-eqiad
Cmjohnson moved T223825: Degraded RAID on restbase-dev1006 from Backlog to Hardware Failure / Troubleshoot on the ops-eqiad board.
Tue, May 28, 2:53 PM · ops-eqiad, Operations
Cmjohnson moved T224260: restbase-dev1006 has a broken disk from Backlog to Hardware Failure / Troubleshoot on the ops-eqiad board.
Tue, May 28, 2:53 PM · Cassandra, RESTBase, Core Platform Team (Security, stability, performance and scalability (TEC1)), Core Platform Team Backlog (Watching / External), Services (watching), Operations, DC-Ops, ops-eqiad
Cmjohnson removed a project from T222391: Gerrit Hardware Upgrade: ops-eqiad.
Tue, May 28, 2:53 PM · Release-Engineering-Team, Release-Engineering-Team-TODO, serviceops, Operations, Gerrit
Cmjohnson closed T224065: cloudvirt1028 - no PS redundancy as Resolved.
Tue, May 28, 2:52 PM · cloud-services-team (Kanban), Operations, ops-eqiad
Cmjohnson moved T224223: decommission lvs100[123456].wikimedia.org from Backlog to Decommission on the ops-eqiad board.
Tue, May 28, 2:51 PM · Traffic, Operations, ops-eqiad, DC-Ops, decommission
Cmjohnson moved T224268: Decommission rhenium from Backlog to Decommission on the ops-eqiad board.
Tue, May 28, 2:51 PM · Operations, ops-eqiad, decommission

May 16 2019

Cmjohnson added a comment to T184293: rack/setup/install lvs101[3-6].

I don't think DC-Ops is holding this task up any longer.

May 16 2019, 9:02 PM · ops-eqiad, Operations, Traffic
Cmjohnson added a comment to T184293: rack/setup/install lvs101[3-6].

lvs1013 idrac is configured and connected to all ports and all switches

May 16 2019, 8:57 PM · ops-eqiad, Operations, Traffic
Cmjohnson added a comment to T184293: rack/setup/install lvs101[3-6].

lvs1014 idrac is configured and is connected to all the switches

May 16 2019, 8:15 PM · ops-eqiad, Operations, Traffic
Cmjohnson updated the task description for T184293: rack/setup/install lvs101[3-6].
May 16 2019, 7:17 PM · ops-eqiad, Operations, Traffic

May 13 2019

Cmjohnson added a comment to T223100: Confirm asset tags for asw2-a6/a7/a8/b5-eqiad.

netbox updated

May 13 2019, 2:58 PM · Operations, ops-eqiad, DC-Ops
Cmjohnson updated subscribers of T223100: Confirm asset tags for asw2-a6/a7/a8/b5-eqiad.

asw2-a6-eqiad Asset tag mismatch for s/n PE3717440136: WMF7322 (Accounting) vs. WMF7232 (Netbox) - Correct asset tag is WMF7322
asw2-a7-eqiad Asset tag mismatch for s/n TA3717230855: WMF7323 (Accounting) vs. WMF7233 (Netbox)- Correct asset tag is WMF7323
asw2-a8-eqiad Asset tag mismatch for s/n PE3717440010: WMF7324 (Accounting) vs. WMF7234 (Netbox) - Correct asset tag is WMF7324
asw2-b5-eqiad Asset tag mismatch for s/n PE3717430036: WMF7329 (Accounting) vs. WMF7239 (Netbox) - Correct asset tag is WMF7329

May 13 2019, 2:57 PM · Operations, ops-eqiad, DC-Ops

May 9 2019

Cmjohnson moved T222731: Storage problems with new host db1133 from Backlog to Hardware Failure / Troubleshoot on the ops-eqiad board.
May 9 2019, 3:07 PM · ops-eqiad, Operations
Cmjohnson added a comment to T222731: Storage problems with new host db1133.

new disk ordered You have successfully submitted request SR990443425.

May 9 2019, 3:07 PM · ops-eqiad, Operations

May 7 2019

Cmjohnson added a comment to T219404: rack/setup/install restbase10[19-27].eqiad.wmnet.

@RobH these are currently powered down

May 7 2019, 6:40 PM · Cassandra, Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (Security, stability, performance and scalability (TEC1)), User-fgiunchedi, User-Eevans, ops-eqiad, Operations, RESTBase
Cmjohnson reassigned T219404: rack/setup/install restbase10[19-27].eqiad.wmnet from Cmjohnson to RobH.

@mobrovac all of the on-site work has been completed, I am assigning to @RobH to finish the installs and hand over to you.

May 7 2019, 6:40 PM · Cassandra, Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (Security, stability, performance and scalability (TEC1)), User-fgiunchedi, User-Eevans, ops-eqiad, Operations, RESTBase
Cmjohnson updated the task description for T219404: rack/setup/install restbase10[19-27].eqiad.wmnet.
May 7 2019, 6:39 PM · Cassandra, Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (Security, stability, performance and scalability (TEC1)), User-fgiunchedi, User-Eevans, ops-eqiad, Operations, RESTBase
Cmjohnson changed the status of T220907: Degraded RAID on ms-be1013 from Open to Stalled.

I drained the flea power and you should not have any issues with the idrac. The server is still out of warranty so not much I can do about the raid at this point in time. I did disabled the network port switch. If you need access again please let me know and I will enable the port.

May 7 2019, 2:47 PM · ops-eqiad, Operations

May 6 2019

Cmjohnson added a comment to T222050: db1107 (eventlogging db master) possibly memory issues.

I now have h/w log entries. I will need the server to be taken offline so I can relocate the DIMM and check to see if the error follows. Unfortunately there are steps involved that need to be done to get Dell to replace.

May 6 2019, 4:49 PM · Analytics, Operations, ops-eqiad, Analytics-EventLogging, DBA
Cmjohnson added a comment to T222127: db1093 (s6 candidate master) went down - broken BBU.

I created a task for this with HPE.

May 6 2019, 4:31 PM · ops-eqiad, Operations, Patch-For-Review, DBA
Cmjohnson added a comment to T219404: rack/setup/install restbase10[19-27].eqiad.wmnet.

@moborvac I haven't had a chance to get to them until this week. I should be able to get them racked and the on-site done this week.

May 6 2019, 3:54 PM · Cassandra, Core Platform Team Kanban (Done with CPT), Services (done), Core Platform Team (Security, stability, performance and scalability (TEC1)), User-fgiunchedi, User-Eevans, ops-eqiad, Operations, RESTBase

May 2 2019

Cmjohnson reassigned T211613: rack/setup/install db11[26-38].eqiad.wmnet from Cmjohnson to RobH.

@RobH all the servers are racked and on-site work has been completed. Some are off and some are in a state that just needs to rebooted.

May 2 2019, 5:43 PM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
Cmjohnson updated the task description for T211613: rack/setup/install db11[26-38].eqiad.wmnet.
May 2 2019, 5:42 PM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
Cmjohnson updated the task description for T211613: rack/setup/install db11[26-38].eqiad.wmnet.
May 2 2019, 4:02 PM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
Cmjohnson updated the task description for T211613: rack/setup/install db11[26-38].eqiad.wmnet.
May 2 2019, 3:22 PM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
Cmjohnson updated the task description for T211613: rack/setup/install db11[26-38].eqiad.wmnet.
May 2 2019, 2:50 PM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
Cmjohnson added a comment to T221047: relocate/reimage cloudvirt1007 with 10G interfaces.

@Andrew the disk has been replaced, all yours to install

May 2 2019, 2:44 PM · Patch-For-Review, ops-eqiad, DC-Ops, Operations, Epic, cloud-services-team (Kanban)

Apr 25 2019

Cmjohnson updated the task description for T211613: rack/setup/install db11[26-38].eqiad.wmnet.
Apr 25 2019, 4:34 PM · Goal, DBA, ops-eqiad, User-Marostegui, Operations
Cmjohnson updated the task description for T211613: rack/setup/install db11[26-38].eqiad.wmnet.
Apr 25 2019, 4:15 PM · Goal, DBA, ops-eqiad, User-Marostegui, Operations