most useful rsync commands
Rsync (Remote Sync) is a most commonly used command for copying and synchronizing files and directories remotely as well as locally in Linux/Unix systems. With the help of rsync command you can copy and synchronize your data remotely and locally across directories, across disks and networks, perform data backups and mirroring between two Linux machines.
Advantages of RSYNC:
• Supports copying links, devices, owners, groups and permissions.
• It’s faster than scp (Secure Copy) because rsync uses remote-update protocol which allows to transfer just the differences between two sets of files. First time, it copies the whole content of a file or a directory from source to destination but from next time, it copies only the changed blocks and bytes to the destination.
• Rsync consumes less bandwidth as it uses compression and decompression method while sending and receiving data both ends.
Options for rsync command:
• -v : verbose
• -r : copies data recursively (but don’t preserve timestamps and permission while transferring data
• -a : archive mode, archive mode allows copying files recursively and it also preserves symbolic links, file permissions, user & group ownerships and timestamps
• -z : compress file data
• -h : human-readable, output numbers in a human-readable format
• -e : used to specify the remote shell protocol.
Examples:
1) Copy/Sync files between two directories in local machine.
[root@localhost source]# pwd
/tmp/source
[root@localhost source]# ls -ltr
total 0
-rw-r–r–. 1 root root 0 Feb 3 22:44 t
-rw-r–r–. 1 root root 0 Feb 3 22:44 t1
-rw-r–r–. 1 root root 0 Feb 3 22:44 t2
[root@localhost source]#
[root@localhost source]# ls -l /tmp/destination/
total 0
There is no files in destination and there is files in the source directory. We use the rsync to copy the files.
[root@localhost source]# rsync -avzh /tmp/source/ /tmp/destination/
sending incremental file list
source/
source/t
source/t1
source/t2
sent 188 bytes received 73 bytes 522.00 bytes/sec
total size is 0 speedup is 0.00
The files are copied to destination folder.
2) Copy/Sync two directories between two servers.
. rsync
Source and destination can be any server details.
Source:
[root@localhost ~]# hostname -I
192.168.41.136 192.168.1.3 192.168.122.1
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# ls -l /tmp/source/
total 0
-rw-r–r–. 1 root root 0 Feb 3 22:44 t
-rw-r–r–. 1 root root 0 Feb 3 22:44 t1
-rw-r–r–. 1 root root 0 Feb 3 22:44 t2
Destination:
[root@localhost ~]# hostname -I
192.168.41.135 192.168.122.1
[root@localhost ~]#
[root@localhost ~]# ls -l /tmp/destination
ls: cannot access /tmp/destination: No such file or directory
Below command copy the files to destination server, also create the directory destination.
[root@localhost ~]# rsync -avhz /tmp/source/ root@192.168.41.135:/tmp/destination/
root@192.168.41.135’s password:
sending incremental file list
created directory /tmp/destination
source/
source/t
source/t1
source/t2
sent 188 bytes received 73 bytes 74.57 bytes/sec
total size is 0 speedup is 0.00
[root@localhost ~]#
3) Rsync over ssh protocol
We can use the ssh protocol to copy the content from one server to another server. Ssh is a secure protocol which perform the encryption of the data when we transfer the data over the wire.
[root@localhost ~]# rsync -avhze ssh /tmp/source/ root@192.168.41.135:/tmp/destination/
root@192.168.41.135’s password:
sending incremental file list
sent 77 bytes received 13 bytes 25.71 bytes/sec
total size is 0 speedup is 0.00
4) To see the progress while transferring the file use below command
[root@localhost ~]# rsync -avhze ssh /tmp/source/ –progress root@192.168.41.135:/tmp/destination/
root@192.168.41.135’s password:
sending incremental file list
source/
source/content
9.40M 100% 107.63MB/s 0:00:00 (xfer#1, to-check=3/5)
sent 32.12K bytes received 35 bytes 9.19K bytes/sec
total size is 9.40M speedup is 292.33
5) Include and exclude options
–exclude=PATTERN exclude files matching PATTERN
–exclude-from=FILE read exclude patterns from FILE
–include=PATTERN don’t exclude files matching PATTERN
–include-from=FILE read include patterns from FILE
–files-from=FILE read list of source-file names from FILE
Here we specify the exclude to exclude some files and use include to give exception for some files from excluding.
[root@localhost ~]# rsync -avhze ssh /tmp/source/ –include ‘t2’ –exclude ‘*’ root@192.168.41.135:/tmp/destination/
root@192.168.41.135’s password:
sending incremental file list
t2
sent 63 bytes received 31 bytes 26.86 bytes/sec
total size is 0 speedup is 0.00
The above command send only the t2 and exclude all the files.
6) “-delete” option
If you want to sync the source and destination and you want to delete files in destination which are not in the source.
[root@localhost source]# rsync -avhz –delete /tmp/source/ root@192.168.41.135:/tmp/destination/
root@192.168.41.135’s password:
sending incremental file list
./
deleting test.txt
deleting t2
sent 65 bytes received 15 bytes 22.86 bytes/sec
total size is 9.40M speedup is 117500.00
7) Set maximum file size for the files to transfer.
We can see the maximum size for the files rsync to transfer. If the files size exceeds more than that files won’t get transferred.
[root@localhost source]# rsync -avhz –max-size=’1024k’ /tmp/source/ root@192.168.41.135:/tmp/destination/
root@192.168.41.135’s password:
sending incremental file list
./
t
t1
we
sent 206 bytes received 72 bytes 111.20 bytes/sec
total size is 868.33M speedup is 3123478.79
The above command don’t transfer the files whose size is more than 1MB.
8) Automatically delete the source files after the transfer to destination.
After transferring the files to destination it removed all files from source folder.
[root@localhost source]# rsync –remove-source-files -avhz /tmp/source/ root@192.168.41.135:/tmp/destination/
root@192.168.41.135’s password:
sending incremental file list
content
donttf
sent 2.95M bytes received 50 bytes 137.14K bytes/sec
total size is 868.33M speedup is 294.50
[root@localhost source]#
[root@localhost source]# ls -l /tmp/source/
total 0
9) Set bandwidth limit to transfer the data.
–bwlimit=KBPS limit I/O bandwidth; KBytes per second
[root@localhost source]# rsync –bwlimit=100 -avhz /tmp/source/ root@192.168.41.135:/tmp/destination/
root@192.168.41.135’s password:
sending incremental file list
./
sent 29 bytes received 15 bytes 12.57 bytes/sec
total size is 0 speedup is 0.00
10) Synchronize only the directory structure not the files.
Use the “-d” option to synchronize only the directory structure not the data files.
[root@localhost source]# rsync -vd /tmp/source/ root@192.168.41.135:/tmp/destination/
root@192.168.41.135’s password:
building file list … done
sent 22 bytes received 12 bytes 5.23 bytes/sec
total size is 0 speedup is 0.00
11) Don’t create new files in destination just copy existing files from source to destination which are already exist in the source.
[root@localhost source]# rsync –existing -avhz /tmp/source/ root@192.168.41.135:/tmp/destination/
root@192.168.41.135’s password:
sending incremental file list
sent 26 bytes received 12 bytes 10.86 bytes/sec
total size is 0 speedup is 0.00
Viswamitra's blog 