Page MenuHomePhabricator

Labs: Salvage, then remove volumes on labstores' raid6
Closed, ResolvedPublic

Description

Once the fsck of store/now is complete, inspect the resulting filesystem to recover files requested by endusers then remove the remaining volumes that live on the raid6 shelves.

Subtasks of this task for individual recovery requests.

Event Timeline

coren created this task.Jun 21 2015, 2:29 PM
coren raised the priority of this task from to Needs Triage.
coren updated the task description. (Show Details)
coren added a subscriber: coren.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 21 2015, 2:29 PM
Blahma added a subscriber: Blahma.Jun 22 2015, 2:48 PM
coren triaged this task as High priority.Jun 22 2015, 5:16 PM
faidon assigned this task to coren.Jun 22 2015, 5:49 PM
faidon set Security to None.

All but one user request has been satisfied, but we need to keep the broken filesystem around until we are done with an ongoing copy to labstore2001.

coren changed the task status from Open to Stalled.Jun 25 2015, 5:30 PM

This is now only pending on the copy to labstore2001 (in progress, from labstore1002)

Blahma removed a subscriber: Blahma.Jun 28 2015, 4:19 PM

https://phabricator.wikimedia.org/P871 is list of files that are irretrievably corrupted.

coren added a comment.Jul 7 2015, 2:45 PM

https://phabricator.wikimedia.org/P871 is list of files that are irretrievably corrupted.

As an important caveat, those are the files for which no data can be recovered, but is not an exhaustive list of possibly damaged files (in particular, files that were being actively written to at and near the time of the crash are often cross-linked with each other in such a way that blocks from both can be read from them). Any log file is automatically suspect, for instance.

coren added a project: Labs-Sprint-105.

Since there is now a copy to recover files from, removing the dependency now and asploding the broken filesystem.

coren moved this task from To Do to Doing on the Labs-Sprint-105 board.Jul 7 2015, 3:04 PM
coren closed this task as Resolved.Jul 7 2015, 3:54 PM

raid6 is dead, long live raid10

coren moved this task from Doing to Done on the Labs-Sprint-105 board.Jul 7 2015, 3:54 PM