From 581b8164f3f4a029917a7efbac2831b905a5123d Mon Sep 17 00:00:00 2001 From: Tom Schraitle Date: Wed, 25 Jan 2023 09:53:29 +0100 Subject: [PATCH] Add topic about backing up with RSnapshot Co-authored-by: Daria Vladykina Co-authored-by: Dmitri Popov <> Co-authored-by: Julia Faltenbacher --- DC-task-create-backup-with-rsnapshot | 11 + xml/task-create-backup-with-rsnapshot.xml | 1042 +++++++++++++++++++++ 2 files changed, 1053 insertions(+) create mode 100644 DC-task-create-backup-with-rsnapshot create mode 100644 xml/task-create-backup-with-rsnapshot.xml diff --git a/DC-task-create-backup-with-rsnapshot b/DC-task-create-backup-with-rsnapshot new file mode 100644 index 000000000..7b814676a --- /dev/null +++ b/DC-task-create-backup-with-rsnapshot @@ -0,0 +1,11 @@ +MAIN="task-create-backup-with-rsnapshot.xml" + +PROFCONDITION="suse-product" +#PROFCONDITION="suse-product;beta" +#PROFCONDITION="community-project" + +DRAFT=1 + +STYLEROOT="/usr/share/xml/docbook/stylesheet/suse2022-ns" +FALLBACK_STYLEROOT="/usr/share/xml/docbook/stylesheet/suse2022-ns" +DOCBOOK5_RNG_URI="urn:x-suse:rnc:v2:geekodoc-flat" diff --git a/xml/task-create-backup-with-rsnapshot.xml b/xml/task-create-backup-with-rsnapshot.xml new file mode 100644 index 000000000..cfb490a11 --- /dev/null +++ b/xml/task-create-backup-with-rsnapshot.xml @@ -0,0 +1,1042 @@ + + + + + %entities; + + rsnapshot'> +]> + + + +
+ + + Creating backups with &rname; + + + https://bugzilla.suse.com/enter_bug.cgi + Smart Docs + Documentation + thomas.schraitle@suse.com + + no + + + rsnapshot + backup + mount + systemd + + Use &rname; to run regular backups on your system + taking incremental snapshots. Prepare your external storage device for + backup, run the backup, and restore snapshots. + + + Backups should be a regular routine for every system administrator and user. + The following document describes how to use &rname; to perform regular backups + based on incremental snapshots. A snapshot captures the state of the files at + the moment the backup is performed. + The article explains how to prepare an external storage device for backup, + configure the data to back up, perform a backup, and how to restore data from + snapshots. + + + + + + +
+ Introduction + The following description shows how to backup system data to an + external storage device using the &rcmd; command. It's suitable for anyone + interested in creating backups of the home, /etc and + other directories. + + &rname;'s core feature applies the following concepts: + + + + Backups have named backup levels and + intervals. + + + + By default, &rname; uses generic backup level names: + alpha, beta, + gamma, and delta. To make + &rname; easier to use, change the defaults to more descriptive names, for + example daily instead of alpha, + weekly instead of beta, and so + on. Use only the levels you need. + + + + The interval of a named backup level defines how many backups you want + to retain. + + + + Each backup contains the named backup level and an ascending + number. For example, the first backup is named + daily.0, the second daily.1 and + so on, until the maximum number of backups is reached. + Same for the other levels like weekly, + monthly or yearly. + + + + Backups are rotated after some time. + This means that older backups are deleted to create space for newer backups. + This is explained in more detail in . + + + +
+ +
+ Requirements + + To use &rname;, you need: + + + + + The rsnapshot package installed on your system. + + + An external USB storage device (for example a hard disk or SSD) reserved + exclusively for backups. The device must have storage capacity for holding + multiple backup snapshots. + + + + The required space depends on the amount of data to back up and the number of + snapshots you want to keep. Rule of thumb is to multiply the size of the + data you want to back up by the number of snapshots. + + + A backup plan. This is a mapping of the backup levels and their + repetition in time. In this guide, the backup level + alpha is defined as a daily backup. + See for more details. + + + +
+ +
+ Understanding backup plan + tom 2023-01-27: I'm aware this is probably a bit more + conceptual than procedural. + Maybe we could extract his section and section "Knowing backup + rotations" to create a RSnapshot concept topic. But I wanted + to keep it in one file. Something for the future... + + + This guide defines the backup level daily + for a daily backup. + Every backup level in your plan needs an interval. Based on the intervals + defined in , + the daily level was defined to create five backups, + starting from Monday. + + + Backup levels and intervals + retain daily 5 +retain weekly 4 +retain monthly 14 +retain yearly 3 + + You would perform the backups like this: + Monday = daily +Tuesday = daily +Wednesday = daily +Thursday = daily +Friday = daily + weekly +Saturday = - +Sunday = - + + In the example above, daily is performed every day + except for Saturday and Sunday. At the end of the week, on a Friday, we + perform a weekly backup with weekly. + + + This configuration could be useful for anyone who wants to have a daily + backups except for the weekend. To keep the backups from previous weeks, + a weekly backup is performed every Friday. + + This procedure is repeated every week. Once a month, you perform an + additional gamma backup. +
+ +
+ Understanding backup rotations + + Rotation means that old backups are deleted so + that the number of backups is constant to the defined interval. The rotation + happens automatically when you run a new backup. + + In this guide, backup is performed manually. It is up to you what backup + level to choose. + + + &rname; allows you to retain a given number of previous backups. + This helps to restore your backup from yesterday, last week, or even three months ago. + + + The following examples demonstrate the backup levels and intervals + mentioned in . + + + When an daily backup is performed for the first + time, &rname; creates: + + daily.0 + + When the backup is performed again, daily.0 is renamed + to daily.1 and the current backup becomes + daily.0: + + daily.0 -> daily.1 +current backup -> daily.0 + + For the next daily backup, the snapshots will look like this: + + daily.1 -> daily.2 +daily.0 -> daily.1 +current backup -> daily.0 + + This is repeated until the maximum number of intervals are reached. When + all numbers are taken (five in this example), the oldest backup is deleted: + + daily.4 -> deleted! +daily.3 -> daily.4 +daily.2 -> daily.3 +daily.1 -> daily.2 +daily.0 -> daily.1 +current backup -> daily.0 + + At some point, it is necessary to switch to the next level. Normally, + it's done when the maximum number of intervals for the defined level is + reached. When the weekly backup is performed for the + first time, &rname; creates the following: + daily.4 -> weekly.0 + + If you repeat it next week, the situation now looks like this: + + weekly.0 -> weekly.1 +daily.4 -> weekly.0 + + The same principle applies to the other backup levels, too. + +
+ +
+ Preparing the external storage device + If you already have a prepared disk for your backups, skip this section. + + On Linux, we recommend formatting the disk with a Linux file system. + + + Beware of data loss + + The following procedure describes how to partition and format the external storage device. + The partitioning operation erases all data on the selected device. Make sure you use on the correct device. + + + + To prepare the external storage device, proceed as follows: + + + Preparing the external storage device + + Identify the external storage disk by capacity or name. + As this disk is used exclusively for backup, it is enough + to create one partition. + Format it with a Linux partition, like XFS. + + + You can perform this step with &yast; or with commands: + + + + + With &yast; + Open &yast; + + + + + Open the Partitioner module. + + + + + Identify the external storage disk. + + + + Create a single partition on your external disk. Format it + with a Linux partition, like XFS. + + + + + + With CLI + Open a terminal + + + + + Enter: + + lsblk -p >/tmp/before.txt + + + + Connect the external device and run: + + lsblk -p >/tmp/after.txt + + + Find the respective disk with: + diff -u /tmp/before.txt /tmp/after.txt + The result will be something like this: + /dev/sdx 8:16 1 1TB 0 disk + The exact device name on the left (/dev/sdx) + and the capacity (1 TB) may vary for you. + + + + + Partition the disk with the command fdisk. + Create one partition. + tom 2023-01-25: Should that be a bit more verbose? + + + + Format the disk with mkfs.xfs using the + device name /dev/sdx: + sudo mkfs.xfs -L backup /dev/sdx + + + + + + + Memorize the UUID value of your external storage device by providing the device name + to the following command. You need this value at a later step: + sudo blkid /dev/sdx +/dev/sdx: ... UUID="2022-05-28-00-28-20-47" ... + + + + After the partition operation is completed, the external storage device has one partition and + is formatted, preferably with a Linux file system like XFS. Furthermore, you + know the UUID value of your disk. + +
+ +
+ Adding automount capabilities + To give the external storage device a constant and easy-to-remember + mount point, use systemd automount. + + + Open a terminal, and follow these steps: + + + Adding the external device to <filename>/etc/fstab</filename> + + + Open the /etc/fstab file: + + sudo vi /etc/fstab + + + Add the following lines: + UUID=2022-05-28-00-28-20-47 /data/backup xfs x-systemd.automount,x-systemd.device-timeout=5s,x-systemd.idle-timeout=300s,noauto 0 0 + + + + An identifier of the disk. This is the UUID value of the external + device from of + . + + + + The mount point. In this example, the external device is mounted on + /data/backup. + + + + + The file system. As in this example XFS was chosen, you need to add + xfs. If you decided to + use a different file system, specify the correct value. + Possible values can be ext2, + ext3, + ext4, or + btrfs. + + + + The mount options. To manage the external disk by + systemd, add the + option. The option + defines, in seconds, + how long systemd should wait for + a device before giving up. The option + defines the idle timeout. + For other available options, see + man 5 systemd.mount. + + + + + + + Create the path for the mount point: + sudo mkdir -p /data/backup + + + Let systemd + pick up your changes: + sudo systemctl daemon-reload + + + Keep in mind that systemd automatically + unmounts the device if the directory is not accessed for a certain period + time. This period is defined using the + option that you specified in + /etc/fstab. +
+ + + +
+ Adapting the &rname; configuration + + Before you can use &rname;, you need to change the configuration file + /etc/rsnapshot.conf to your needs. + + + Pay attention to tabs + The &rname; configuration file uses tabs (code + 0x07) to separate the configuration keys from their + values. To avoid errors in the configuration file, make sure that the editor you are using + does not replace tabs with spaces. + + + After you changed the configuration file, it is recommended to run + sudo rsnapshot configtest. + + + + + + Create a backup of the &rname; configuration file. + This is just a precaution in case you need to reconfigure the file or want + to see the differences to your changes: + + sudo cp /etc/rsnapshot.conf /etc/rsnapshot.conf.default + + + + Create a subdirectory /etc/rsnapshot.d. Then add the following files and scripts: + + sudo install --directory --mode=0500 /etc/rsnapshot.d + + As you may also store sensitive information, the directory is created with very + restrictive permissions. Only &rootuser; can read and change into this + directory. + + + + Open the file /etc/rsnapshot.conf: + sudo vi /etc/rsnapshot.conf + + + Change the configuration file: + + + Set the home of your backup: + snapshot_root /data/backup + + + + Set the backup names and levels, see also for more information. + One example could be: + + retain daily 5 +retain weekly 4 +retain monthly 12 +retain yearly 3 + + This configuration keeps seven daily backups from + Monday to Sunday, four weekly backups, + 12 monthly backups, and three + yearly backups. + + + + Set the verbosity level. It may be useful to raise the verbosity + level to 3 when you use the program for the first + time: + verbose 2 + By default, the value is set to 2, which means it + prints only errors and warnings. Raising it to 3, gives you + all the shell commands that are being executed. + Alternatively, you can also temporarily raise the verbosity + level by adding the option to the &rcmd; + command. + + + Change the default options for rsync, the sync + tool for &rname;: + rsync_long_args -aAX --numeric-ids + + The options turn on the archive mode (), + which preserves everything, except ACLs and extended + attributes. You can enable the storage of both with the options + (ACLs) and (extended attributes). + The option transfers numeric group and user IDs instead + of user and group names. For more details, see the man page. + + + + + Set exclude parameters. By default, &rname; backs up everything. To exclude files that end with a + tilde or with .bak, use the following rule: + + exclude *~ +exclude *.bak + + + Set an exclude file. To exclude files matching multiple patterns, add them to the exclude file instead of listing them with the + option. Then you can add the exclude file as follows: + exclude_file /etc/rsnapshot.d/excludes.txt + + + Define what you want to back up. In most cases, + /home and /etc should be + included in your backup. You can add more paths, depending on your needs. + In this example, a few other subdirectories from + /var were added: + backup_exec /bin/date "+ backup of /home started at %c" +backup /home localhost/ + +backup_exec /bin/date "+ backup of /etc started at %c" +backup /etc localhost/ + +backup_exec /bin/date "+ backup of /var/lib started at %c" +backup /var/lib/ca-certificates/ localhost/ +backup /var/lib/smartmontools/ localhost/ +backup /var/adm/backup/rpmdb/ localhost/ +backup /var/lib/rpm/ localhost/ + + + Optional lines starting with can be used for debugging and printing the + time and date when the respective backup part was started. + You can also use them to execute scripts before the real backup happens. + + + + Defines the backup and contains the backup path and + the host name. + + + Defines the backup path that needs to be stored in the backup. + + + Defines the host name. You can replace localhost with your real host name. + + + This value is important when you want to back up + different systems. A descriptive value makes the systems easier to identify. + + + + + + Save the file. + + + + + If you have set the option in + , create the file /etc/rsnapshot.d/excludes.txt: + + sudo vi /etc/rsnapshot.d/home-excludes.txt + + Each line in this file contains a pattern that excludes a file or + directory. Lines that start with #, + ; and blank lines are ignored. + + This file is used to specify paths that belong to the + operating system or are virtual paths like /run, + /proc, or /sys. + For example: + # General excludes that is usually not worth to backup: +/bin +/dev/ +/mnt/ +/proc/ +/run/ +/tmp/ +/sys +/usr + + + Amend or create the file + /etc/rsnapshot.d/excludes.txt. Add lines to exclude + from the home directory. You can use wildcards like * + and ? to match a group of files or directories. The + following lines exclude paths from any home directories that are usually not + worth storing. Keep in mind, as we use * in the + path, it matches every user: + # Personal excludes for every user +/home/*/.cache/ +/home/*/.local/tmp/ +/home/*/Downloads/ + You can also add user-specific paths. The + following example excludes a directory under Documents for + the user &exampleuser;: + # Specific user excludes: +/home/&exampleuser_plain;/Documents/tests + + + Check the configuration file: + sudo rsnapshot configtest + You should get: + Syntax OK + + If you receive an error, &rname; usually lists the line number where + the error occurred. Fix this error and repeat this step. + + + + Do a test run: + sudo rsnapshot -t daily + If you run it for the first time, you should get something like this: + echo 7439 > /var/run/rsnapshot.pid +mkdir -m 0700 -p /data/backup/ +mkdir -m 0755 -p /data/backup/daily.0/ +/usr/bin/rsync -a --delete --numeric-ids --relative --delete-excluded \ + --exclude=*~ --exclude=*.bak \ + --exclude-from=/etc/rsnapshot.d/excludes.txt /home/ \ + /data/backup/daily.0/localhost/ +mkdir -m 0755 -p /data/backup/daily.0/ +/usr/bin/rsync -a --delete --numeric-ids --relative --delete-excluded \ + --exclude=*~ --exclude=*.bak \ + --exclude-from=/etc/rsnapshot.d/excludes.txt /etc/ \ + /data/backup/daily.0/localhost/ +mkdir -m 0755 -p /data/backup/daily.0/ +/usr/bin/rsync -a --delete --numeric-ids --relative --delete-excluded \ + --exclude=*~ --exclude=*.bak \ + --exclude-from=/etc/rsnapshot.d/excludes.txt /usr/ \ + /data/backup/daily.0/localhost/ +touch /data/backup/daily.0/ + + +
+ +
+ Running the first backup + After you have configured the external device and + changed the &rname; configuration file, it is time to run the backup. + + + Connect the external storage device. + + + Mount the device manually: + sudo systemctl start data-backup.automount +● data-backup.automount + Loaded: loaded (/etc/fstab; generated) + Active: active (running) since Tue 2023-01-24 07:31:53 CET; 9h ago + Triggers: ● data-backup.mount + Where: /data/backup + Docs: man:fstab(5) + man:systemd-fstab-generator(8) + + + + Run the first backup and use the backup level + daily: + sudo rsnapshot -v daily + The backup will start. If it runs for the first time, it takes a + considerable amount of time. All subsequent backups are completed + faster. + + + + After the backup is finished, you can find it at + /data/backup/daily.0/localhost/. + + + + To unmount the external storage device, use the following command: + sudo systemctl stop data-backup.automount + + + Before unplugging the storage device, wait for confirmation that it has been successfully unmounted. + + +
+ +
+ Running consecutive backups + + For any consecutive backups, follow these steps: + + + + Connect the external device. + + + Mount the device: + sudo systemctl start data-backup.automount + + + + Perform backup. Because you perform the backup manually, you need to + decide which is next. + In most cases, it would be daily: + + sudo rsnapshot daily + When you want to switch to weekly, + run: + sudo rsnapshot weekly + + The same applies to the other backup levels. + + + + To unmount the external storage device, use the following command: + sudo systemctl stop data-backup.automount + + + Before unplugging the storage device, wait for confirmation that it has been successfully unmounted. + + +
+ +
+ Restoring a backup + When the external storage device is mounted, all the existing snapshots are + visible under the snapshot's root, the backup root directory. In our case it + is /data/backup/. The latest backup of the host can be + found under /data/backup/daily.0/localhost. The entire + directory tree looks like this: + + File structure of the backup root directory + /data/backup/ +├── daily.0 +│ └── localhost +│ ├── etc +│ └── home +├── [... daily ...] +├── weekly.0 +│ └── localhost +│ ├── etc +│ └── home +├── [... weekly ...] +├── monthly.0 +│ └── localhost +│ ├── etc +│ └── home +├── [... monthly ...] +└── [... yearly ...] + + + + In the following subsections, we use the command + rsync to restore the backups. This command is also used internally by &rname;. + By default, rsync does not take into account ACLs and + extended attributes of a file system. To preserve these as well, you need to + pass the options and to the + command. + +
+ Restoring files + + To restore a single file from the most recent backup, + use the rsync command as follows: + + sudo rsync -av --acls --xattrs \ + /data/backup/daily.0/localhost/home/&exampleuser_plain;/test-a.txt \ + /home/&exampleuser_plain;/ +sending incremental file list +test-a.txt +[...] + + This restores the file test-a.txt to the home of + user &exampleuser;. + + + Restoring multiple file files of the same base directory is done in a similar manner. + Specify all the desired files as follows: + + sudo rsync -av --acls --xattrs \ + /data/backup/daily.0/localhost/home/&exampleuser_plain;/test-a.txt \ + /data/backup/daily.0/localhost/home/&exampleuser_plain;/test-b.txt \ + /home/&exampleuser_plain;/ + This restores the file test-a.txt and + test-b.txt to the home of user &exampleuser;. + If you have files with different directories, you need to run + rsync for each file or directory.. +
+ +
+ Restoring directories + Restoring directories is similar to restoring + files. However, pay special attention to a slash (/) at + the end of the source path or + not: + + + + + Omitting the slash from the source path + + Instructs rsync to restore the + complete directory: + + + sudo rsync -av --acls --xattrs \ + /data/backup/daily.0/localhost/home/&exampleuser_plain;/Documents \ + /home/&exampleuser_plain;/ +sending incremental file list +Documents/ +Documents/foo/ +Documents/foo/bar/ +[...] + + + + Adding a slash to the source path + Instructs rsync to restore the content of the directory, skipping + the directory itself: + + sudo rsync -av --acls --xattrs \ + /data/backup/daily.0/localhost/home/&exampleuser_plain;/Documents/ \ + /home/&exampleuser_plain;/ +sending incremental file list +./ +foo/ +foo/bar/ +[...] + + +
+ +
+ Restoring a complete home directory + This is the same as restoring a directory. + sudo rsync -av --acls --xattrs \ + /data/backup/daily.0/localhost/home/&exampleuser_plain; \ + /home/ +sending incremental file list +&exampleuser_plain;/ +&exampleuser_plain;/.bash_history +&exampleuser_plain;/.bashrc +[...] + + This restores the complete home directory of user &exampleuser;. If you want + to know what data is restored, use the option as follows: + + sudo rsync --log-file=/var/log/home-&exampleuser_plain;.log [... rest ...] + The file /var/log/home-&exampleuser_plain;.log + contains an exhaustive list of all files that have been restored. +
+
+ +
+ Troubleshooting + + + + When I run rsnapshot daily, I get: + ERROR: /data/backup/ does not exist. +ERROR: rsnapshot refuses to create snapshot_root when no_create_root is enabled + + + The external storage device is probably not mounted. Mount + the device, and check if the disk is mounted successfully with the command + df: + /dev/sdc1 xfs 1,0T 845G 156G 85% /data/backup + + + + + I get the following error: + ERROR: /etc/rsnapshot.conf on line 23: +ERROR: snapshot_root /data/backup/ - missing tabs to separate words - \ + change spaces to tabs. + + + There is probably a space between a keyword and its value. Open + your editor and position the cursor on line 23. Remove the space between + keyword and value and press . + + + + + I checked the config with rsnapshot configtest and + it is correct. However, I still get an error: + /usr/bin/rsync syntax or usage error. Does this version of rsync support --link-dest? + + + There may be a typo in one of the options for + rsync. Check and + carefully and compare the options with + the man page. + + + +
+ + + +
+ Related topics + + + + + + + +
+