Page MenuHomePhabricator

Develop, package, deploy and document a single file recovery utility
Closed, ResolvedPublic

Description

An simple, fast, easy-to-use utility to recover one or a few files from backup and decrypt and transfer to the local filesystem and/or the swift production cluster is a must.

The application or script should probably live on ms-backup hosts, and ask for a title, hash, or other methods to identify 1 or a few backed up files, either interactively or on command line/with an api, and download or import those files into swift.

This should be on a maintained git repo and packaged and deployed into production.
Its usage should be very simple, as it may be used during an outage, but anyway should be extensively documented (both the general backups architecture and its operation) on wikitech.

Event Timeline

jcrespo changed the task status from Open to In Progress.Jan 25 2022, 11:35 AM
jcrespo triaged this task as High priority.

Change 772824 had a related patch set uploaded (by Jcrespo; author: Jcrespo):

[operations/software/mediabackups@master] Initial release

https://gerrit.wikimedia.org/r/772824

Change 772831 had a related patch set uploaded (by Jcrespo; author: Jcrespo):

[operations/puppet@production] mediabackups: Install package and its dependencies through .deb

https://gerrit.wikimedia.org/r/772831

Change 772832 had a related patch set uploaded (by Jcrespo; author: Jcrespo):

[integration/config@master] Add operations/software/mediabackups to CI tests

https://gerrit.wikimedia.org/r/772832

Change 772885 had a related patch set uploaded (by Jcrespo; author: Jcrespo):

[operations/puppet@production] mediabackups: Add reference key for file decryption on recovery config

https://gerrit.wikimedia.org/r/772885

Change 772885 merged by Jcrespo:

[operations/puppet@production] mediabackups: Add reference to key for decryption on recovery config too

https://gerrit.wikimedia.org/r/772885

Change 772824 merged by Jcrespo:

[operations/software/mediabackups@master] Initial release of mediabackups software

https://gerrit.wikimedia.org/r/772824

Change 772831 merged by Jcrespo:

[operations/puppet@production] mediabackups: Install package and its dependencies through .deb

https://gerrit.wikimedia.org/r/772831

Change 772832 merged by jenkins-bot:

[integration/config@master] Add operations/software/mediabackups to CI tests

https://gerrit.wikimedia.org/r/772832

Recovery procedure (for a small number of files, not yet fully automated) documented at:

https://wikitech.wikimedia.org/wiki/Media_storage/Backups#Recovering_files

^ @LSobanski check if it is somewhat understandable.

@jcrespo Looks good, I made a few formatting changes, see what you think. There's one sentence that may be incomplete: "If there are errors (e.g. download or decryption fails)."

After review, I consider the ticket as solved- with the following wanted features to be handled at separate tickets:

  • Listing and deletion
  • Non-interactive command line/API version
  • Uploading directly to MediaWiki/Swift
  • Batching multiple files (e.g. from a list)
  • Test environment