Page MenuHomePhabricator
Create Task
Maniphest T186535

OfflineUncorrectableSector on mw1256 sda
Closed, ResolvedPublic
Actions

Description

This message was generated by the smartd daemon running on:

host name:  mw1256
DNS domain: eqiad.wmnet

The following warning/error was logged by the smartd daemon:

Device: /dev/sda [SAT], 1 Offline uncorrectable sectors

Device info:
WDC WD5003ABYX-18WERA0, 500 GB

For details see host's SYSLOG.

You can also use the smartctl utility for further investigation.
Another message will be sent in 24 hours if the problem persists.

Event Timeline

fgiunchedi created this task.Feb 5 2018, 4:32 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 5 2018, 4:32 PM
• Cmjohnson added subscribers: Joe, • Cmjohnson.Feb 7 2018, 10:32 PM

@Joe The disk is out of warranty but I have kajillion 500GB disks lying around. The disk can be replaced anytime but a re-install will be needed. Let me know once the server is depooled and powered off.

• Cmjohnson moved this task from Backlog to Up next on the ops-eqiad board.Feb 7 2018, 10:32 PM
Stashbot subscribed.Feb 8 2018, 7:27 AM

Mentioned in SAL (#wikimedia-operations) [2018-02-08T07:27:33Z] <_joe_> depooled mw1256 from traffic, scap (faulty disk, T186535); now powering it off

RobH assigned this task to • Cmjohnson.Feb 8 2018, 7:03 PM
RobH triaged this task as Medium priority.
RobH subscribed.

SAL states depooled & looks like its ready to go, powered down.

In T186535#3954891, @Stashbot wrote:

Mentioned in SAL (#wikimedia-operations) [2018-02-08T07:27:33Z] <_joe_> depooled mw1256 from traffic, scap (faulty disk, T186535); now powering it off

• Cmjohnson added a comment.Feb 13 2018, 4:09 PM

The disk has been replaced and needs a re-install

• Cmjohnson moved this task from Up next to High Priority Task on the ops-eqiad board.Feb 13 2018, 4:11 PM
fgiunchedi added a comment.Feb 13 2018, 4:37 PM

I tried to access mw1256.mgmt.eqiad.wmnet at 10.65.2.106 though that ends up being lead's console:

Debian GNU/Linux 8 lead ttyS1

lead login:
Stashbot added a comment.Feb 13 2018, 5:54 PM

Mentioned in SAL (#wikimedia-operations) [2018-02-13T17:54:24Z] <godog> repool mw1256 after disk swap - T186535

• Cmjohnson moved this task from High Priority Task to Blocked on the ops-eqiad board.Feb 22 2018, 6:03 PM
• Cmjohnson closed this task as Resolved.Mar 14 2018, 7:06 PM

This has been completed. resolving

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits