Unix tools introduced. Today: rsync

rsync is a very cool tool that can be used to copy files between hosts or between directories on the same host. Like the term ‘sync’ suggests the copy process can be controlled into great detail to modulate rsync’s behavior.  Take a look at the available options under: https://linux.die.net/man/1/rsync

This is my list of cool options.  I start with the most basic usage. The following command can be used to copy, and later on sync two directories.

rsync -avn /source/dir /target/dir  

The command ‘archives’ file attributes (-a) and displays some status info (-v).

In the given form, the command only does a dry-run (-n). To execute the command remove the -n.

The command uses the short form of --archive (-a) which translates to (-rlptgoD).

  • -r – recursive copy
  • -l – copy symlinks as symlinks
  • -p – set target permissions to be the same as the source
  • -t – set target mtime to be the same as the source. Use this to support fast incremental updates based on mtime.
  • -g – set target group to be the same as the source
  • -o – set target owner to be the same as the source
  • -D – if remote user is superuser this recreates devices and other special files.

More cool options

Move

--remove-source-files This will remove copied files from source.

Update

--update This forces rsync to skip any files which exist on the destination and have a modified time that is newer than the source file.

Delete

--delete Delete files on target that do not exist in source tree.

Backup

--backup Make a backup of modified or removed files on target.

--backup-dir=date +%Y.%m.%d Specify a backup dir on target.

What to copy?

--min-size=1 Do not copy empty files. This can be particularly interesting if you have corrupted files in the source.

--max-size=100K Copy only small files. Can be used to handle small and large files differently.

--existing Only override files that already exist on the target. Do not create new files on target.

--ignore-existing Only copy files that do not exist on target.

--exclude-from Define excludes in a file.

Scheduling, Bandwidth and Performance

--time-limit Ends rsync after a certain time limit.

--stop-at=y-m-dTh:m Ends rsync at a specific time.

--partial Allows partial copies in case of interruptions.

--bwlimit=100 Limits bandwidth Specify KBytes/second. Good option if transfer of large files is required.

Output

  • -h output numbers in a human-readable format.
  • --progress display progress.
  • -i log change info.
  • --log-file= define a log file.
  • --quiet no output.
  •  -v Output status info. You can add more ‘v’.
  • Forgot to log any progress info? Use the following command to see what rsync is about to do.
     ls -l /proc/$(pidof rsync)/fd/*

 

Leave a Reply