I recently have to transfer a big file between two GNU/Linux hosts and through a slow connection network.
I'll show here what I did to achieve it.
For the example I use a private ip address (192.168.1.200) but it obviously works with a public ip.
The operation will consist in dividing the big file into several small ones. So in case of a network failure we will be able to resume the transfer where it has stopped.
root@host:~# apt-get install rsync openssh-server
user@SOURCE:~$ split MyBigFile.mkv -d -b 5M
user@SOURCE:~$ ls -lh -rw-r--r-- 1 std std 5,0M 8 déc. 23:47 x00 -rw-r--r-- 1 std std 5,0M 8 déc. 23:47 x01 [...] -rw-r--r-- 1 std std 5,0M 8 déc. 23:48 x9481 -rw-r--r-- 1 std std 5,0M 8 déc. 23:48 x9482
Now we can transfer the files to our destination host. In case of failure the rsync software will be able to resume the transfer from the last file transferred and thus avoid starting from the beginning.
user@DESTINATION:~$ rsync --bwlimit=40k -a -v --rsh='ssh -p 22' --stats --progress user@192.168.1.200:/home/SOURCE/x* /home/DESTINATION/
user@DESTINATION:~$ cd /home/DESTINATION/
user@DESTINATION:~$ cat x* > glory.41.720p.hdtv.x264-verum.mkv
user@DESTINATION:~$ ls -lh glory.41.720p.hdtv.x264-verum.mkv -rw-r--r-- 1 toi toi 2,9G 8 déc. 23:45 glory.41.720p.hdtv.x264-verum.mkv
Contact :