HomePhabricator

[WIP] Create framework to transfer files over the LAN

Description

[WIP] Create framework to transfer files over the LAN

The transfer method must:

  • Be as fast as the network bandwidth allows it, with configurable throttling
  • Be easy to use (take automatic decisions when safe)
  • Allow for both single files and entire directories to be synced. The directories can be of thousands of small files
  • Keep the original permissions and ownership
  • Have in-place checks to avoid doing damaging stuff
  • Allow encryption
  • Allow configurable compression
  • Allos configurable resource taking (e.g. number of CPUs)
  • Hash contents before and after the copy to check it has been done successfuly
  • Allow multicast-like transfers from 1 server to many
  • Report the status at any time, and if it fails, why
  • Handle the firewall automatically
  • Not require a constantly open port or service

The current code is just the barebones, it has to be integrated
with volan's packages for remote code execution.

Because the above, rsync is not enough. We have to give a look to
multi-thread FTP, tar + socat with user encryption, and bittorent.

Change-Id: I2add041f7884e61c81436119a23e14fd94db7d5d

Details

Provenance
jcrespoAuthored on Dec 9 2016, 6:46 PM
RduranCommitted on May 17 2018, 11:31 AM
Parents
rOSMD5d0fa1fcf46f: Add mysql.py wrapper
Branches
Unknown
Tags
Unknown
References
refs/changes/56/433556/1
ChangeId
I2add041f7884e61c81436119a23e14fd94db7d5d