I’ve been having trouble with my Gladys backup for several weeks, but I hadn’t taken the time to do the necessary checks.
In fact, every night when the backup is triggered, Gladys freezes (no dashboard display, no scene triggering, etc.), this lasts about 2 hours, then I receive the message « Gladys just restarted » in Telegram (this is the result of a scene with the trigger « Gladys starts »), and everything works again.
With Claude’s help this morning, I explored the issue. At one point in his analysis, I replied that I had « a medium-sized base (a few gigabytes) », but I’m not sure I answered correctly. In the history of my backups, I see this:
As a first action, Claude suggested I modify the RAM allocated to Gladys (with the command sudo docker update --memory="6g" --memory-swap="8g" gladys), so that my issue no longer causes a freeze for several hours. And indeed, when I manually restarted a backup, the crash and restart took 3-4 minutes. It’s better to not block my home automation, but it doesn’t solve the backup failure issue yet 
I’m running on an Intel NUC5PPYB mini-PC, N3700 processor 4 cores 1.6GHz, 8GB of RAM, under Ubuntu 22.04, and Gladys was installed with the standard « docker run » command. I don’t have anything else on this server besides Gladys.
Here is Claude’s final report:
Memory Peak / OOM-kill during GladysPlus backup
Summary
Every GladysPlus backup (automatic 2h-4h or manual) causes the Gladys container’s RAM to spike to the point of triggering an OOM-kill of the Node process, followed by an automatic restart of the container.
Environment
- Gladys in Docker on Mini PC/NUC
- Total host RAM: 7.7 GB (8 GB non-extensible)
- Database: medium size (a few GB)
gladyscontainer manually limited to 6 GB of RAM (docker update --memory) to prevent OOM from destabilizing the entire host (swap-thrashing observed before this limit was implemented)
Reproduction
Manual backup triggered at 09:36.
- 09:36:01 - RSS before backup: 4388.77 MB, Heap 233.97/392.50 MB
- 09:36:01 - DuckDB before backup: Total 2248.75 MB (BASE_TABLE: 2246.00 MB, IN_MEMORY_TABLE: 2.75 MB)
- 09:36:04 - Opening a second DuckDB instance dedicated to backup (
duckDbCreateBackupInstance,memory_limit=1024MB,threads=2,READ_ONLY) to export to a Parquet folder - 09:38 - Container RAM: ~6 GB (limit reached)
- 09:40 -
Out of memory: Killed process ... (MainThread) - 09:41 - Container restarted, Gladys functional again
Without the 6 GB limit manually set at the Docker level, the same phenomenon in real conditions (automatic nightly backup) caused a swap-thrashing episode on the entire host from 02:23 to 04:44 (dashboard and scenes unusable), the RSS of the process having reached 6774.54 MB before kill (dmesg: Out of memory: Killed process 1542 (MainThread) total-vm:30424440kB, anon-rss:6774536kB).
Hypothesis
The new DuckDB instance created for the backup (limited to 1024 MB) seems to be added to the memory already occupied by the main DuckDB instance (~2.25 GB), rather than sharing/respecting a global memory budget of the process. On a machine where the process is already running at ~4.4 GB at rest, adding ~1 GB more is enough to exceed the available RAM.
Questions for maintainers
- Is it normal that
BASE_TABLEremains almost entirely in memory (2.25 GB) at rest, outside of backup, for a medium-sized base? - Can the
memory_limit=1024MBof the backup instance be lowered, or can the backup process stream/process in batches instead of loading so much in memory, to stay within a reasonable total envelope even on hardware with 8 GB of RAM?
(or someone else!)