How do you backup?

[email protected]

Sure, but that's not in their answer.

[email protected]

They didn't provide an rsync example until later in the post, the comment about not supporting differential backups is in reference to using rsync itself, which is incorrect, because rsync does support differential backups.

I agree with you that not doing differential backups is a problem, I'm simply commenting that this is not a drawback of using rsync, it's an implementation problem on the user's part. It would be like somebody saying "I like my Rav4, it's just problematic because I don't go to the grocery store with it" and someone else saying "that's a big drawback, the grocery store has a lot of important items and you need to be able to go to it". While true, it's based on a faulty premise, because of course a Rav4 can go to the grocery store like any other car, it's a non-issue to begin with. OP just needs to fix their backup script to start doing differential backups.

[email protected]

Right now, nothing (but my dotfiles/etc configs live on several machines)

Before, I used Restic (incremental, encrypted backup over network), which I really liked. I think I should set it up again.

[email protected]

My one and only purpose was to warn them that their "drawback" is more of a gator pit. It's noble that you're here defending rsync's honor, but maybe let them know instead? My preferred backup tool has "don't eat my data" mode on by default.

? Offline

The restore process takes forever and sometimes fails. Last time I was forced to try every daily backup to several days before last backup to find one that could actually be restored. I have switched to borg (using Pika Backup for desktop and Borgmatic for servers). No restores have failed since.

[email protected]

All my configs are in gitlab or a self hosted forgejo server and all files are in seafile or a self hosted service running on proxmox. Then I use proxmox backup server on a storage VPS for off-site backup

[email protected]

Using timeshift. Very, very easy, works great.

[email protected]

I have a server with a RAID-1 array, that makes daily, weekly, and monthly read only btrfs snapshots. The whole thing (sans snapshots) is sync'd with syncthing to two rPi's in two different geographic locations.

I know neither raid nor syncthing are "real" backup solutions, but with so many copies of the files living in so many locations (in addition to my phone, laptop, etc.) I'm reasonably confident its a decent solution.

[email protected]

Borg Backup, whenever I feel like it - usually monthly.

[email protected]

I use external drive for my important data and if my system is borked (which never happen to me) I just reinstall the OS

[email protected]

Mostly, with some folders excepted (e.g. my Nextcloud folder)

[email protected]

Thanks for your feedback, maybe I should look into switching

[email protected]

Offline Backup on 2 separate HDD/SSD
Backup on HDD within my desktop pc
Backup offsite with restic to Hetzner Storage Box

[email protected]

External drives are more prone to damage and failures, both because they're more likely to be dropped/bumped/spilled on etc, and because of generally cheaper construction compared to internal drives. In the case of SSDs the difference might be negligible, but I suggest you at least make a copy on another "cold" external drive if the data is actually important

[email protected]

I use Duplicacy to backup to my TrueNAS server. Crucial data like documents are backed up a second time to my GDrive, also using Duplicacy. Sadly it's a paid solution, but it works great for me.

[email protected]

Borg daily to the local drive then copied across to a USB drive, then weekly to cloud storage. Script is triggered by daily runs of topgrade before I do any updates

[email protected]

I created a script that I dropped into /etc/cron.hourly which does the following:

Use rsync to mirror my root partition to a btrfs partition on another hard drive (which only updates modified files).
Use btrfs subvolume snapshot to create a snapshot of that mirror (which only uses additional storage for modified files).
Moves "old" snapshots into a trash directory so I can delete them later if I want to save space.

It is as follows:

#!/usr/bin/env python
from datetime import datetime, timedelta
import os
import pathlib
import shutil
import subprocess
import sys

import portalocker

DATETIME_FORMAT = '%Y-%m-%d-%H%M'
BACKUP_DIRECTORY = pathlib.Path('/backups/internal')
MIRROR_DIRECTORY = BACKUP_DIRECTORY / 'mirror'
SNAPSHOT_DIRECTORY = BACKUP_DIRECTORY / 'snapshots'
TRASH_DIRECTORY = BACKUP_DIRECTORY / 'trash'

EXCLUDED = [
    '/backups',
    '/dev',
    '/media',
    '/lost+found',
    '/mnt',
    '/nix',
    '/proc',
    '/run',
    '/sys',
    '/tmp',
    '/var',

    '/home/*/.cache',
    '/home/*/.local/share/flatpak',
    '/home/*/.local/share/Trash',
    '/home/*/.steam',
    '/home/*/Downloads',
    '/home/*/Trash',
]

OPTIONS = [
    '-avAXH',
    '--delete',
    '--delete-excluded',
    '--numeric-ids',
    '--relative',
    '--progress',
]

def execute(command, *options):
    print('>', command, *options)
    subprocess.run((command,) + options).check_returncode()

execute(
    '/usr/bin/mount',
    '-o', 'rw,remount',
    BACKUP_DIRECTORY,
)

try:
    with portalocker.Lock(os.path.join(BACKUP_DIRECTORY,'lock')):
        execute(
            '/usr/bin/rsync',
            '/',
            MIRROR_DIRECTORY,
            *(
                OPTIONS
                +
                [f'--exclude={excluded_path}' for excluded_path in EXCLUDED]
            )
        )

        execute(
            '/usr/bin/btrfs',
            'subvolume',
            'snapshot',
            '-r',
            MIRROR_DIRECTORY,
            SNAPSHOT_DIRECTORY / datetime.now().strftime(DATETIME_FORMAT),
        )

        snapshot_datetimes = sorted(
            (
                datetime.strptime(filename, DATETIME_FORMAT)
                for filename in os.listdir(SNAPSHOT_DIRECTORY)
            ),
        )

        # Keep the last 24 hours of snapshot_datetimes
        one_day_ago = datetime.now() - timedelta(days=1)
        while snapshot_datetimes and snapshot_datetimes[-1] >= one_day_ago:
            snapshot_datetimes.pop()

        # Helper function for selecting all of the snapshot_datetimes for a given day/month
        def prune_all_with(get_metric):
            this = get_metric(snapshot_datetimes[-1])
            snapshot_datetimes.pop()
            while snapshot_datetimes and get_metric(snapshot_datetimes[-1]) == this:
                snapshot = SNAPSHOT_DIRECTORY / snapshot_datetimes[-1].strftime(DATETIME_FORMAT)
                snapshot_datetimes.pop()
                execute('/usr/bin/btrfs', 'property', 'set', '-ts', snapshot, 'ro', 'false')
                shutil.move(snapshot, TRASH_DIRECTORY)

        # Keep daily snapshot_datetimes for the last month
        last_daily_to_keep = datetime.now().date() - timedelta(days=30)
        while snapshot_datetimes and snapshot_datetimes[-1].date() >= last_daily_to_keep:
            prune_all_with(lambda x: x.date())

        # Keep weekly snapshot_datetimes for the last three month
        last_weekly_to_keep = datetime.now().date() - timedelta(days=90)
        while snapshot_datetimes and snapshot_datetimes[-1].date() >= last_weekly_to_keep:
            prune_all_with(lambda x: x.date().isocalendar().week)

        # Keep monthly snapshot_datetimes forever
        while snapshot_datetimes:
            prune_all_with(lambda x: x.date().month)
except portalocker.AlreadyLocked:
    sys.exit('Backup already in progress.')
finally:
    execute(
        '/usr/bin/mount',
        '-o', 'ro,remount',
        BACKUP_DIRECTORY,
    )

[email protected]

I use Pika Backup (GUI that uses Borg Backup on the backend) to back up my desktop to my home server daily, then overnight that server has a daily backup using Borg to a Hetzner Storage Box. It's easy to set it and forget it (other than maybe verifying the backups every once in a while), and having that off site back up gives me peace of mind.

[email protected]

How do you make sure the disk spins down? Is unmounting enough?

[email protected]

Do you use some kind of encryption on the VPS?

agnos.is Forums

How do you backup?