Gladys container crash / memory full at night

[pierre-gilles]

Ok I made a build where I created a special « read-only » connection to the DB for the backup, which I close once the backup is finished.

In my tests, it seems to work but I’m curious to see your test

It’s available on the same tag.

[Checconio]

Ça a l’air pas mal !

2026-01-19T16:21:29+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before backup - RSS: 419.78 MB, Heap: 241.69/277.03 MB
2026-01-19T16:21:29+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before backup - Total: 2.50 MB
2026-01-19T16:21:29+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:21:29+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T16:21:29+0100 <info> system.isDocker.js:13 (System.isDocker) System.isDocker: This system is running inside Docker.
2026-01-19T16:21:29+0100 <info> gateway.backup.js:79 (Gateway.backup) Gateway backup: Locking SQLite Database
2026-01-19T16:21:29+0100 <info> gateway.backup.js:84 () Backup attempt n°1 : Cleaning backup folder
2026-01-19T16:21:29+0100 <info> gateway.backup.js:88 () Starting Gateway backup in folder /var/lib/gladysassistant/backups/gladys-db-backup-2026-1-19-16-21-29.db
2026-01-19T16:21:29+0100 <info> gateway.backup.js:90 () Gateway backup: Unlocking Database
2026-01-19T16:21:29+0100 <info> gateway.backup.js:96 (Gateway.backup) Gateway backup : SQLite file size is 2mb.
2026-01-19T16:21:29+0100 <info> gateway.backup.js:97 (Gateway.backup) Gateway backup : Backing up DuckDB into a Parquet folder /var/lib/gladysassistant/backups/gladys-db-backup_2026-1-19-16-21-29_parquet_folder
2026-01-19T16:22:33+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory after export database to parquet - RSS: 2811.57 MB, Heap: 226.00/249.53 MB
2026-01-19T16:22:33+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory after export database to parquet - Total: 2.75 MB
2026-01-19T16:22:33+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:22:33+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:22:33+0100 <info> gateway.backup.js:111 (Gateway.backup) Gateway backup: Closing DuckDB backup instance to release memory
2026-01-19T16:22:33+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory after closing backup instance - RSS: 2811.45 MB, Heap: 226.47/249.53 MB
2026-01-19T16:22:33+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory after closing backup instance - Total: 2.75 MB
2026-01-19T16:22:33+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:22:33+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:22:33+0100 <info> gateway.backup.js:116 (Gateway.backup) Gateway backup: Compressing backup
2026-01-19T16:23:01+0100 <info> gateway.backup.js:122 (Gateway.backup) Gateway backup: Encrypting backup
2026-01-19T16:23:03+0100 <info> gateway.backup.js:129 (Gateway.backup) Gateway backup: Uploading backup, size of encrypted backup = 443mb. Path = /var/lib/gladysassistant/backups/gladys-db-backup-2026-1-19-16-21-29.tar.gz.enc
2026-01-19T16:23:03+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 0 - RSS: 2614.45 MB, Heap: 232.99/256.53 MB
2026-01-19T16:23:03+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 0 - Total: 2.75 MB
2026-01-19T16:23:03+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:03+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:04+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 1 - RSS: 444.46 MB, Heap: 221.74/259.78 MB
2026-01-19T16:23:04+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 1 - Total: 2.75 MB
2026-01-19T16:23:04+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:04+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:05+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 2 - RSS: 417.40 MB, Heap: 210.64/254.78 MB
2026-01-19T16:23:05+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 2 - Total: 2.75 MB
2026-01-19T16:23:05+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:05+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:06+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 3 - RSS: 432.16 MB, Heap: 211.50/246.59 MB
2026-01-19T16:23:06+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 3 - Total: 2.75 MB
2026-01-19T16:23:06+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:06+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:06+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 4 - RSS: 451.83 MB, Heap: 212.45/246.59 MB
2026-01-19T16:23:06+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 4 - Total: 2.75 MB
2026-01-19T16:23:06+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:06+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:07+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 5 - RSS: 472.02 MB, Heap: 207.63/246.59 MB
2026-01-19T16:23:07+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 5 - Total: 2.75 MB
2026-01-19T16:23:07+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:07+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:08+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 6 - RSS: 492.15 MB, Heap: 208.47/246.59 MB
2026-01-19T16:23:08+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 6 - Total: 2.75 MB
2026-01-19T16:23:08+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:08+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:08+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 7 - RSS: 471.50 MB, Heap: 207.71/246.84 MB
2026-01-19T16:23:08+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 7 - Total: 2.75 MB
2026-01-19T16:23:08+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:08+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:09+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 8 - RSS: 491.62 MB, Heap: 208.85/246.84 MB
2026-01-19T16:23:09+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 8 - Total: 2.75 MB
2026-01-19T16:23:09+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:09+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:10+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 9 - RSS: 472.57 MB, Heap: 208.25/246.84 MB
2026-01-19T16:23:10+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 9 - Total: 2.75 MB
2026-01-19T16:23:10+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:10+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:11+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 10 - RSS: 492.07 MB, Heap: 209.87/246.84 MB
2026-01-19T16:23:11+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 10 - Total: 2.75 MB
2026-01-19T16:23:11+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:11+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:12+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 11 - RSS: 474.23 MB, Heap: 211.14/246.59 MB
2026-01-19T16:23:12+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 11 - Total: 2.75 MB
2026-01-19T16:23:12+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:12+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:13+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 12 - RSS: 494.98 MB, Heap: 207.42/246.59 MB
2026-01-19T16:23:13+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 12 - Total: 2.75 MB
2026-01-19T16:23:13+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:13+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:14+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 13 - RSS: 455.16 MB, Heap: 207.80/246.59 MB
2026-01-19T16:23:14+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 13 - Total: 2.75 MB
2026-01-19T16:23:14+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:14+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:15+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 14 - RSS: 495.15 MB, Heap: 209.56/246.59 MB
2026-01-19T16:23:15+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 14 - Total: 2.75 MB
2026-01-19T16:23:15+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:15+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:16+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 15 - RSS: 474.92 MB, Heap: 209.01/246.84 MB
2026-01-19T16:23:16+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 15 - Total: 2.75 MB
2026-01-19T16:23:16+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:16+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:17+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 16 - RSS: 494.77 MB, Heap: 210.85/246.84 MB
2026-01-19T16:23:17+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 16 - Total: 2.75 MB
2026-01-19T16:23:17+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:17+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:19+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 17 - RSS: 454.84 MB, Heap: 211.07/246.84 MB
2026-01-19T16:23:19+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 17 - Total: 2.75 MB
2026-01-19T16:23:19+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:19+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:21+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 18 - RSS: 495.13 MB, Heap: 207.39/246.84 MB
2026-01-19T16:23:21+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 18 - Total: 2.75 MB
2026-01-19T16:23:21+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:21+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:23+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 19 - RSS: 495.12 MB, Heap: 209.18/246.84 MB
2026-01-19T16:23:23+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 19 - Total: 2.75 MB
2026-01-19T16:23:23+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:23+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:25+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 20 - RSS: 474.77 MB, Heap: 207.70/246.59 MB
2026-01-19T16:23:25+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 20 - Total: 2.75 MB
2026-01-19T16:23:25+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:25+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:27+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 21 - RSS: 494.90 MB, Heap: 209.08/246.59 MB
2026-01-19T16:23:27+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 21 - Total: 2.75 MB
2026-01-19T16:23:27+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:27+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:28+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 22 - RSS: 454.86 MB, Heap: 210.96/246.84 MB
2026-01-19T16:23:28+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 22 - Total: 2.75 MB
2026-01-19T16:23:28+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:28+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB
2026-01-19T16:23:29+0100 <info> gateway.backup.js:175 (Gateway.backup) Gladys backup uploaded with success to Gladys Gateway.
2026-01-19T16:23:29+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory after upload - RSS: 475.01 MB, Heap: 213.18/246.84 MB
2026-01-19T16:23:29+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory after upload - Total: 2.75 MB
2026-01-19T16:23:29+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 1.00 MB
2026-01-19T16:23:29+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.75 MB

[pierre-gilles]

Nice! I’ll clean up the PR and integrate that into Gladys then

[Checconio]

I’m reverting to the latest production version and I’ll continue to monitor whether my other underlying issue reappears.

That way, I’m trying to provide some details.

[pierre-gilles]

The fix is live in Gladys Assistant v4.67 :

Gladys Assistant v4.67 : Nouvelle intégration Nuki

Salut à tous, Je suis heureux de vous présenter une nouvelle version de Gladys ! Nouveautés & améliorations Nouvelle intégration : Nuki Vous pouvez désormais connecter vos serrures connectées Nuki à Gladys et les intégrer dans vos scénarios et tableaux de bords. Merci @ProtZ ! HomeKit Mise à jour de la dépendance HAP vers la dernière version stable, pour une meilleure compatibilité et plus de fiabilité avec HomeKit. Merci @bertrandda ! Gladys…