How to use Rsync to exclude Files and Directories in Data Transfer

Getting Started

Rsync is a useful Linux command-line tool that synchronizes and copies files and directories. You can use the tool to synchronize data locally between directories and drives, or between two remote computers.

The basic rsync commands on Linux synchronize everything from a location you specify. In many backup scenarios, you may want to exclude specific files, directories, or file types.

Follow the examples in this guide to learn how to exclude files or directories with rsync. We’ll list the most common exclusion use cases to cover your daily rsync usage.

  • Prerequisites User with sudo or root privileges
  • Access to a command line/terminal window
  • Rsync installed on the system

How

the rsync exclusion option works The

-exclude option with the rsync command uses paths relative to the source directory. Append the -exclude option to the rsync command, followed by the relative path to a directory or file.

The basic syntax for the rsync exclude option looks like this: rsync

[OPTIONS] -exclude ‘file_or_directory’ source/ destination/ Replace source/ with the

directory name you want to use as the source for data transfer to another location. Replace destination/ with

the

directory name that rsync will use as the destination location for your data. If the directory does not exist, rsync creates one for you and transfers the files to that directory.

To back up files to

a remote location, follow our other guide to learn how to copy or transfer files with rsync via SSH.

Exclude a specific

file

In the rsync exclusion examples below, we’ll use the –a (file) and –v (verbose) (-av) options. The -a option synchronizes directories recursively while maintaining group permissions, symbolic links, ownership, and settings. The -v flag is optional and prints the progress and status of the rsync command.

To exclude a file while transferring the contents of a folder with rsync, specify the file and relative path.

For example

: rsync -av -exclude ‘testfile1.txt’ sourcedir/ destinationdir/ This command

allows you to copy the files to destinationdir from sourcedir but exclude

testfile1.txt.

As you can see in the output, testfile1.txt is not listed.

To do this, run this command:

sudo rm -rf destinationdir

Exclude a specific directory

Now that you know how to use the rsync –exclude file command, use the tool in the same way to exclude a directory.

Specify a directory you want to exclude (instead of a file name): rsync -av –

exclude

‘dir1’ sourcedir/ destinationdir

/ This command copied the contents of sourcedir to destinationdir and excluded dir1, as seen in the output

. Exclude files or directories

based on

a pattern

Use an asterisk * (wildcard) when defining a file or directory name to exclude anything that matches the pattern

. To exclude files that

begin with test, run this command

: rsync -av -exclude ‘test*’ sourcedir/

destinationdir/ All files and directories that match this pattern will be excluded from the transfer

.

You can also use the wildcard in a similar subject to exclude all directories that end with a specific pattern. Run this command to exclude all directories ending with

the

number 3:

rsync -av -exclude ‘*3’ sourcedir/

destinationdir/ You can use an asterisk before and after a pattern to further refine the -exclude criterion

.

Exclude a specific file type

The rsync tool allows you to exclude certain file types when synchronizing data. Use an asterisk * followed by the extension of the file type you want to exclude.

For example, you might want to back up a directory that contains many .iso files that you don’t need to back up.

To exclude a specific file type, in this case .iso, run this command:

rsync -av -exclude ‘*.iso’ sourcedir/

destinationdir/ The output shows that rsync did not transfer .iso files. We add the contents of the source directory in the image for comparison.

Exclude files by

size

During file transfers, you can specify the minimum or maximum size of files that you want to exclude. If the source directory contains many large files that you do not want to back up, use the –max-size=size_in_mb_or_gb option. Replace the size_in_mb_or_gb with the desired size.

For example, to exclude

all files larger than 500 MB, run this command:

rsync -av -max-size=500m sourcedir/

destinationdir/ On the other hand, to exclude files smaller than a specific size, use the –min-size=size_in_mb_or_gb option

.

For example, you want to transfer a directory with images, but there are many thumbnail files. If all your images are larger than 1 MB, you can exclude all files smaller than that size.

To do this, run this command:

rsync -av -min-size=1m sourcedir/ destinationdir/

Do not use the -exclude rsync with option when setting the minimum or maximum file size. Exclude multiple files or directories Add multiple -exclude options to exclude multiple files or

directories

. You can combine any rsync -exclude folder and rsync -exclude file(s) command to transfer only the data you need. Any of the commands we talked about above can be used on one line.

The following command will exclude all files with the .txt extension, as well

as dir3 and dir4 directories: rsync -av -exclude ‘*.txt’ -exclude ‘dir3’ -exclude ‘dir4’ sourcedir/ destinationdir/

You can add as many -exclude entries as you need. To keep the command tidy, you can specify files and directories in the -exclude option using curly braces. Separate patterns using a comma.

For example, we can shorten the above command like this:

rsync -av -exclude={‘*.txt’,’dir3′,’dir4′} sourcedir/

destinationdir/ The result shows that the listed files and directories are excluded from the transfer.

Exclude files and directories from a list When you need to exclude a large number

of different files and directories

, you can use the rsync –exclude-from flag command. To do this, create a text file with the name of the files and directories that you want to exclude. Then pass the file name to the –exlude-from option.

The

command looks like this:

rsync -av -exclude-from={‘list.txt’} sourcedir/

destinationdir/ The rsync tool ignores all files and directories that it lists in the file. You can add any pattern we use in this guide.

The sample file we used in the previous command contains the following patterns:

Conclusion

This guide showed you all the examples of rsync exclusion you need when transferring data with this tool. You can combine the patterns to customize the command and exclude multiple files and directories with rsync.

Contact US