Get Mozilla Firefox
From time to time, I need to transfer large files between two GNU/Linux hosts over a relatively slow connection. To do this, I split the large files into several smaller ones, which I reassemble once all the smaller files have been transferred. The advantage is manifold: if the copy fails during transfer, I don't have to transfer the whole thing again, and what's more, I can make the copy in several stages. Let's see how to do this in detail below!
In this example, I'm using a private IP address (192.168.1.200), but it obviously works with a public IP.
As indicated in the introduction, the operation consists in dividing the large file into several smaller ones.
root@host:~# apt-get update && apt-get install rsync openssh-server
user@SOURCE:~$ du -sh MyBigFile.mkv
-rw-r--r-- 1 user std 2,9G 8 déc. 12:12 MyBigFile.mkv
To split the large file into several smaller files, we'll use the cleverly named: split command.
user@SOURCE:~$ split MyBigFile.mkv -d -b 5M
user@SOURCE:~$ ls -lh
-rw-r--r-- 1 user std 5,0M 8 déc. 23:47 x00
-rw-r--r-- 1 user std 5,0M 8 déc. 23:47 x01
[...]
-rw-r--r-- 1 user std 5,0M 8 déc. 23:48 x9481
-rw-r--r-- 1 user std 5,0M 8 déc. 23:48 x9482
We can now transfer the files to our destination host. In the event of failure, the rsync software will be able to resume the transfer from the last file transferred, thus avoiding the need to start all over again.
user@DESTINATION:~$ rsync --bwlimit=400k -a -v --rsh='ssh -p 22' --stats --progress user@192.168.1.200:/home/SOURCE/x* /home/DESTINATION/
user@DESTINATION:~$ cd /home/DESTINATION/
user@DESTINATION:~$ cat x* > glory.41.720p.hdtv.x264-verum.mkv
user@DESTINATION:~$ ls -lh glory.41.720p.hdtv.x264-verum.mkv
-rw-r--r-- 1 user std 2,9G 8 déc. 23:45 glory.41.720p.hdtv.x264-verum.mkv
Contact :
