Rsync Introduction

Jason Moore

Here is how to use Rsync with examples.

Introduction

Rsync stands for remote synchronization and it transfers and syncs files. It can do this locally and remotely. It is a very good tool. Though it has a learning curve, it is not hard to pick up. Its main use is to copy files and directories between two different computers. It can look at files and only send what has been changed. It can preserve all kinds of links and metadata.    

Installing Rsync     

If you do not already have it installed on your system, you will need to install it. I am running Fedora. If you are running another distribution, use whatever package manager you have to install it.

 

On Fedora run:

dnf update -y

 

This will update your files. Then:

dnf install rsync -y

 

This will install rsync to your system if it is not already there.

 

Now run:

which rsync

 

This will show you where it is installed on your system

Then run:

rsync –version

 

That shows you the version you have.

 

Copying Files

Copying files is really easy. It is:

 

rsync -v source destination

 

The -v option means output will be given verbosely

Source is the full path of the source file unless you are in its directory already.

Destination should be the full path unless it is in your current path too.

It looks like this:

 

rsync -v program1.cpp Documents

 

In the above example I was already in the directory of the file I wanted to copy. You should do that when you can. I transferred it to the Documents folder.

Another example that is slightly different:

 

Rsync -av /home/jason/documents /home/jason/Writing/

 

This command copies all the files in Documents to my Writing folder.

You can use the ls command to look and make sure everything is transferred as expected.

 

Ls Writing/

 

There are many reasons to make copies of your files. Backing up important files to another remote location is something we should all do more.

 

Whenever you do a file transfer, it is a good idea to switch to that location and make sure it is copied over. Doing this a few times will instill confidence in your command line abilities.

 

The Trailing /

The trailing slash at the end of a path dictates whether rsync will copy the contents of a directory or the entire directory with the folder included. Excluding the / from the source path copies the directory to the source destination.

 

# This command will copy the Writing directory and its contents to the backup drive

Rsync -avz /home/jason/Writing /path/BackupDrive/

 

# This command will only copy the files in the Writing directory to the backup drive.

Rsync -avz /home/jason/Writing/ /path/BackupDrive

 

This is a small difference but it is very important to get right.

 

Copying Contents of Directories

It is often very useful to copy entire directories at once. It is easy to do this. Use:

 

Rsync -av     /source/     /destination/

 

Just use the full paths of the source and destination

So, something like this should get the job done:

 

Rsync -av /home/jason/Documents/ /home/jason/Backup/

 

Copying Directories to other Directories

If we want to copy a folder to another folder then we do this:

 

Rsync -av /home/jason/Documents /home/jason/Backup/

 

You should look inside the directory to make sure you typed the command over correctly. You should see the folder nested in there.

 

Copying A File Remotely

Rsync lets you connect to different machines. This makes copying files to other machines an easy practice. You will need:

  1. File path from local machine
  2. IP address of remote machine
  3. File path on remote machine
  4. Root access to remote machine

The command will look something like this depending on what you need to do:

 

Rsync -v /path/from/local/machine     [email protected]:/root/remote/path

 

Copying Directory To Another Drive

This is very handy and gives you better protection. It is also easy to implement. 

 

Rsync -av /home/jason/Writing /path/BackupDrive

 

As usual, go and look to make sure everything happened the way you expect. After a while, you will not feel the need to do this.

 

Copying Directories Remotely

Rsync can handle remote directories just as easily as single files. When you run this command, you will be asked for its password. So, be prepared on this front. The command looks like this:

 

Rsync -av   /local/path   [email protected]:/root/remote/path/

 

Compressing Files

Rsync can compress files that it tries to transfer. This will speed up a transfer. If your transfer is very small, you will not see a difference. However, if you are doing lots of video, for example, this will be of great benefit. Do it like this:

 

Rsync -avz /home/jason/video /path/BackupDrive/

 

This command will copy the Video folder over to my backup drive.

 

Monitoring Your Progress

If we are doing a long transfer, we can monitor the progress. I like statistics so this is useful for me. The command looks like this:

 

Rsync -avz –info=progress2  /home/jason/Video /path/BackupDrive/

 

This will give you the results of your transfer.

 

Syncing Directories

Syncing directories is easily done. Keep in mind that sometimes files will be deleted and they will be gone. So, use this command after careful consideration. We use the –delete option with the regular command plus source and destination paths. This will look at the source directory and then make the destination directory match it. It looks like this:

 

Rsync -aP –delete /home/jason/Writing/ /path/BackupDrive/Writing/

 

Excluding Files and Directories

Rsync can easily look the other way during a command if you want it to. So, if I want to exclude a subfolder of my Writing folder, it will do that. Here is how.

 

Rsync -avzP –exclude=Algebra /home/jason/Writing /path/BackupDrive

 

We can also exclude files from a transfer or sync operation. If I want to exclude .mp3 files it looks like this:

 

Rsync -avzP –exclude=*.mp3 /home/jason/Music/ /path/BackupDrive

 

Options

  • -a = –archive mode and equal to several other flags at once. It tells rsync to sync recursively,transfer special and block devices, preserve symbolic links,modification times, groups, ownership, and permissions
  • -z = –compress. This option compresses the data that is sent to the destination machine.
  • -P = –partial and –progress. Using this option shows a progress bar during the transfer and keeps track of partially transferred files.
  • –delete. When you use this option, it will delete extra files from the destination folder that are not in the source folder. It is how you mirror directories.
  • -q or –quiet. Use this when you don’t want to see error messages
  • -e. Use this when you want to choose the remote shell to use