Skip to content

Config file and profiles

Martin Pool edited this page Feb 25, 2024 · 2 revisions

Issue: https://github.com/sourcefrog/conserve/issues/163

Have a toml, yaml, or json5 file that specifies what should be backed up to where.

Specify multiple "profiles", each with a source and destination.

With profiles configured then backup, diff, restore, versions, etc, all optionally take a --profile, -p NAME instead of an archive and source. (But, maybe restore should still require the destination to be specified.)

Example

[profile.home]
source = "/home/mbp"
archive = "/backup/machine/home_mbp.c6"
period = "10 min"  # make a new backup this often; all times are parsed with humantime
retain_hourly = "4 weeks" 
retain_daily = "12 months" # delete all but the first backup in each UTC day that is older than 12 months
retain_weekly = "3 years"
exclude = [ "target/debug", "target/release" ]
exclude_from = [ "~/src/backup-scripts/home-exclude" ]

[mirror.home_s3]
source_archive = "/backup/machine/home_mbp.c6"
dest_archive = "s3://bucket.example/home_mbp.c6"
aws_credentials = "~/.secret/conserve-s3-credentials.json"  # or maybe point to a keyring or read from env/imds
retain_daily = "3 years" # this one expires slower than the local backup

[mirror.home_flash]
source_archive = "/backup/machine/home_mbp.c6"
dest_archive = "/volumes/conserve/home_mbp.c6"
if_archive_mounted = true # silently skip if the destination does not exist

conserve config subcommands

conserve config path prints the path.

conserve config add interactively configure a new profile?

set to set values

conserve run

conserve run will run through all profiles (or selected profiles).

For each profile that has a period configured, look at how much time has passed since the start of the last version. If it's due for a new backup, make one.

After making backups, before expiry, it will mirror from each source to its destination.

Then, for every backup or mirror that has expiry configured, find any versions that are older, and delete them.

You can configure with --action=backup,expire,mirror,... to run only some of these.

run --once does at most one backup/expiry for each profile.

run --continuous will sleep after each backup until it's time for the next one, although arguably this is overlapping with the job of systemd etc.

systemd integration

Maybe it would fit more naturally in systemd to have each profile be an individual unit...? Although, that would make it harder to mirror after each backup is complete.

Validation

Maybe we want to say that each archive should be validated at a certain period or after each backup...?

If we allow multiple series per archive then maybe validation should be done once per archive, not per source.

Clone this wiki locally