Page MenuHomePhabricator

ms-be1040 - disk issues
Closed, ResolvedPublic

Description

puppet run on ms-be1040 fails because of disk issues:

https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=ms-be1040&service=puppet+last+run

[mkfs-/dev/sdc1]/returns: mkfs.xfs: /dev/sdc1 appears to contain an existing filesystem (linux_raid_member).

Exec[mkfs-/dev/sdc1]/returns: change from 'notrun' to ['0'] failed: 'mkfs -t xfs -L swift-sdc1 -m crc=1 -m finobt=0 -i size=512 /dev/sdc1' returned 1

Exec[xfs_label-/dev/sdb3]/returns: change from 'notrun' to ['0'] failed: 'xfs_admin -L swift-sdb3 /dev/sdb3' returned 1

Exec[xfs_label-/dev/sdb4]/returns: change from 'notrun' to ['0'] failed: 'xfs_admin -L swift-sdb4 /dev/sdb4' returned 1

Event Timeline

Dzahn created this task.Aug 5 2019, 8:12 PM
Restricted Application added a project: Operations. · View Herald TranscriptAug 5 2019, 8:12 PM
wiki_willy added a subscriber: wiki_willy.

Confirmed server is under warranty thru March 2021.

Dzahn removed a subscriber: Dzahn.Aug 5 2019, 9:24 PM

It looks like to me the host came back up with the wrong disk ordering, (sdc should be sdb), I'll try rebooting the host

fgiunchedi closed this task as Resolved.Aug 6 2019, 10:05 AM

Looks like we're back, and this problem is an instance of T163673: Some swift disks wrongly mounted on 5 ms-be hosts

ms-be1040:~$ pat
Warning: Downgrading to PSON for future requests
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Info: Caching catalog for ms-be1040.eqiad.wmnet
Info: Applying configuration version '1565085878'
Notice: Applied catalog in 11.52 seconds