Backup solutions for Raspberry Pi

Configuration
1

Finally receiving my Raspberry Pi 4 + SSD :rocket:, I did a complete reinstall of Gladys and started installing other services on my Raspberry.

I know that the Gladys database is backed up in Gladys Plus :muscle:, but I wonder about other data/services.

How do you manage backups?

  • occasional disk copy?
  • manual copy script (rsync) to another machine (in the cloud for example)
  • use of a backup tool (syncthings, rpibackup, …)
2

What exactly is not currently kept in Gladys Plus?

Are the Zigbee2mqtt/MQTT files missing in general?

I wonder if these services should be able to recover from these backups during a DB backup restore (re-create containers/files)

Because in the end, there is nothing in these files that is not in the DB, right?

Otherwise, on my servers I use rclone: https://rclone.org/ for all file backups to the cloud. You can connect this to anything (Google Drive, Amazon S3, Backblaze, Dropbox, etc.)

3

Only the ZigbeeMqtt files. The MQTT containers will be recreated with the same parameters.

For Z2M, there are two things to back up:

  • the configuration.yaml file. But I think we can recreate it identically from a database backup.
  • the database.db file which contains all the devices on the network.
    Source: FAQ | Zigbee2MQTT

I’m creating an issue to track this: The Zigbee2mqtt service should be able to be restored from a backup · Issue #1378 · GladysAssistant/Gladys · GitHub

My question was also more general (backing up files outside of Gladys). I’m going to test RClone, thanks :pray:

4

Hello, on my side I use a fairly basic script that performs SCP of compressed gladys files to a server. I also integrated a ping to the Healthchecks service which allows me to be alerted by a Gotify server in case of failure.

5

Ok! So in the Zigbee2mqtt integration, ideally there should be a kind of « startup routine Â» that checks that the container is properly launched, etc., and if the famous configuration.yaml file is not there, recreate it from the DB, we agree?

I don’t necessarily think that this is related to the Gladys Plus todo, so if we develop this on the Zigbee2mqtt side, it will allow someone who does their own backups to restart an instance from their DB :slight_smile:

6

Hello,
I’m following up on this message from @Albenss because I would also like to set up a backup in this way. Since I don’t use GladysPlus, what folders/files should I back up so I can restore them if my Raspberry crashes and I have to start from a fresh Gladys Raspbian image?

Thanks in advance for the answer!

7

The /var/lib/gladysassistant folder contains everything :slight_smile:

However, be careful when performing a « hot Â» backup. Gladys writes to an SQLite database in WAL mode, and therefore there are 3 DB files. If you back up the 3 files sequentially, you will have a corrupted DB

The best approach is to use the SQLite3 CLI to duplicate the DB into a single file before backing it up

8

Thanks for these details.
I imagine I can also do a « docker container stop gladys Â» before saving the folder in question, then restart with a « docker container start gladys Â»?

9

Would it be worth making the sqlite and rclone CLI available on the gladys image?

10

Absolutely @StephaneB :slight_smile:

docker start gladys

It also works

The sqlite3 CLI is already present in the Gladys Docker image (for Gladys Plus backups). It can be reused if needed.

As for Rclone, in my opinion, it’s clearly up to the user to set it up if they wish. It’s a very developer-oriented tool to configure. If the user wants to install it, they can, but I don’t think it’s up to us to include it (neither in the Raspberry Pi OS image, and even less in the Docker image). We stay minimalist on dependencies :slight_smile:

11

Thanks @pierre-gilles for the confirmation

12

If this can give you ideas, on my side I take snapshots of the file system (btrfs), which I back up in an encrypted manner via Restic on different media (cloud/NAS/external disk, depending on the data).
Snapshots allow me to back up the DBs without service interruption and with little risk of data loss. Be careful, however, it’s an advanced technique that requires playing with volumes/partitions/file system, the risk of data loss is very real.

Moreover, I see that you mention Syncthing. It’s a great software, but it only does data replication, not backup! It may seem close, but these are two completely different things. If a file becomes corrupted on a medium, this corruption is likely to spread to other synchronized storage. Same for file deletion.
To be avoided, therefore, for backup.