Gladys container crash / memory full at night

Configuration
21

Because I’m outputting a real-time graph of VA power ^^

But reducing it to a single payload doesn’t seem to have changed the memory issue.

I’ll keep digging.

22

Could you still try switching to a lower frequency, just to see if that’s the issue? Every second seems very aggressive to me! ^^

A lot happens when a sensor value is received — in particular Gladys checks that no scene should be triggered. Here, given the number of values you receive for each payload every second, that’s heavy for Gladys, I think.

23

I also had this issue myself during the night from Saturday to Sunday at 3:00 a.m.
I had a spike in RAM usage up to the maximum and then it crashed.
I don’t have any more details, but if it happens again I’ll investigate further.

2 Likes
24

I just switched to 60 seconds to see if it’s better

1 Like
25

The issue is still present.

So it’s not coming from the requests that my Linky module makes.
I’ll see how I can dig a bit deeper at the process level.

1 Like
26

You should check whether you have another very aggressive device?

27

I do have motion detectors that send fairly frequent reports.

I’ll look into limiting anything that appears frequently in the logs.

1 Like
28

Okay, I dug deeper into the container’s memory.

After a few days of reporting, I was able to identify that the memory is being saturated by DuckDB or at least by a query to DuckDB.
I ran an analysis script on each Node.js DuckDB query to identify which query to DuckDB was causing the problem.
Probably related to the fact that my database is fairly large.

I did a big cleanup of metrics that didn’t need historical data to slim down the database.
Since then, it seems much more stable to me, but as a result I might not know what the problem was before ^^

If the memory increase doesn’t happen again, I’ll restore my database from before the cleanup to reproduce the issue.

29

It’s probably the graphical display query; if at any point your database has the issue, don’t hesitate to export it and send it to me so I can run some tests on it!

30

Yes, I’m waiting for the issue to happen again regarding the large memory spike.

I also noticed that during a Gateway backup, memory increases (which is completely normal), but it is not released after the backup finishes.

Example backup from this morning:

2026-01-19T03:51:19+0100 <info> gateway.checkIfBackupNeeded.js:44 (Timeout._onTimeout) Starting backup!
2026-01-19T03:51:19+0100 <info> system.isDocker.js:13 (System.isDocker) System.isDocker: This system is running inside Docker.
2026-01-19T03:51:19+0100 <info> gateway.backup.js:55 (Gateway.backup) Gateway backup: Locking SQLite Database
2026-01-19T03:51:19+0100 <info> gateway.backup.js:60 () Backup attempt n°1 : Cleaning backup folder
2026-01-19T03:51:19+0100 <info> gateway.backup.js:64 () Starting Gateway backup in folder /var/lib/gladysassistant/backups/gladys-db-backup-2026-1-19-3-51-19.db
2026-01-19T03:51:19+0100 <info> gateway.backup.js:66 () Gateway backup: Unlocking Database
2026-01-19T03:51:19+0100 <info> gateway.backup.js:72 (Gateway.backup) Gateway backup : SQLite file size is 2mb.
2026-01-19T03:51:19+0100 <info> gateway.backup.js:73 (Gateway.backup) Gateway backup : Backing up DuckDB into a Parquet folder /var/lib/gladysassistant/backups/gladys-db-backup_2026-1-19-3-51-19_parquet_folder
2026-01-19T03:52:14+0100 <info> gateway.backup.js:82 (Gateway.backup) Gateway backup: Compressing backup
2026-01-19T03:52:42+0100 <info> gateway.backup.js:88 (Gateway.backup) Gateway backup: Encrypting backup
2026-01-19T03:52:43+0100 <info> gateway.backup.js:95 (Gateway.backup) Gateway backup: Uploading backup, size of encrypted backup = 443mb. Path = /var/lib/gladysassistant/backups/gladys-db-backup-2026-1-19-3-51-19.tar.gz.enc
2026-01-19T03:53:04+0100 <info> gateway.backup.js:139 (Gateway.backup) Gladys backup uploaded with success to Gladys Gateway.

image

31

Are you sure you’re measuring the right thing? How do you measure memory usage?

32

I didn’t go into the memory details there; I saw that this morning by chance.

It’s the overall memory, but the increase is right at the time of the backup.

image

It starts at 3:51, exactly at the start of the SQLite and DuckDB backup in the logs.

33

Yes yes, the increase does indeed occur at the time of the backup.

I just wanted to understand how you actually measure RAM usage?

Depending on the method used, you won’t necessarily see the RAM being « freed » after the backup, and that’s perfectly normal on Linux.

Some tools show the total RAM used by including the cache and buffers, which can give the impression that no RAM has been freed, whereas the RAM is indeed freed and immediately reusable.

But it’s also possible that there’s an issue with memory being released — that’s why the technical details would help a lot :wink:

34

Ah yes I understand, the command that is used to retrieve my graph:

free | grep 'Mem' | awk '{print ($3/$2)*100}'

So, I just ran the test

Before backup:

image

5 min after backup

image

36

I ran tests locally at home with large DuckDB databases, and I don’t see anything that jumps out at me.

That said, real-world conditions can be different, so I made a special build that adds logs to the production version to see where the RAM usage is coming from :slightly_smiling_face:

With this build, you’ll see the RAM usage of Node.js + DuckDB before the backup, during the backup, and after the backup!

The build is available under the tag below — can you test it?

I only made an x64 build, since if I remember correctly you’re on a mini-PC?

(No arm64 build to save time)

gladysassistant/gladys:add-memory-logs-during-backup
37

Parfait, je viens de faire le test, je n’ai ouvert aucun dashboard après le démarrage du container.

2026-01-19T15:16:11+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before backup - RSS: 375.19 MB, Heap: 214.96/234.36 MB
2026-01-19T15:16:11+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before backup - Total: 3.50 MB
2026-01-19T15:16:11+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 0.75 MB
2026-01-19T15:16:11+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:16:11+0100 <info> system.isDocker.js:13 (System.isDocker) System.isDocker: This system is running inside Docker.
2026-01-19T15:16:11+0100 <info> gateway.backup.js:79 (Gateway.backup) Gateway backup: Locking SQLite Database
2026-01-19T15:16:11+0100 <info> gateway.backup.js:84 () Backup attempt n°1 : Cleaning backup folder
2026-01-19T15:16:11+0100 <info> gateway.backup.js:88 () Starting Gateway backup in folder /var/lib/gladysassistant/backups/gladys-db-backup-2026-1-19-15-16-11.db
2026-01-19T15:16:11+0100 <info> gateway.backup.js:90 () Gateway backup: Unlocking Database
2026-01-19T15:16:11+0100 <info> gateway.backup.js:96 (Gateway.backup) Gateway backup : SQLite file size is 2mb.
2026-01-19T15:16:11+0100 <info> gateway.backup.js:97 (Gateway.backup) Gateway backup : Backing up DuckDB into a Parquet folder /var/lib/gladysassistant/backups/gladys-db-backup_2026-1-19-15-16-11_parquet_folder
2026-01-19T15:17:06+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory after export database to parquet - RSS: 3574.57 MB, Heap: 212.67/227.86 MB
2026-01-19T15:17:08+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory after export database to parquet - Total: 2153.00 MB
2026-01-19T15:17:08+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:08+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:08+0100 <info> gateway.backup.js:108 (Gateway.backup) Gateway backup: Compressing backup
2026-01-19T15:17:36+0100 <info> gateway.backup.js:114 (Gateway.backup) Gateway backup: Encrypting backup
2026-01-19T15:17:37+0100 <info> gateway.backup.js:121 (Gateway.backup) Gateway backup: Uploading backup, size of encrypted backup = 444mb. Path = /var/lib/gladysassistant/backups/gladys-db-backup-2026-1-19-15-16-11.tar.gz.enc
2026-01-19T15:17:38+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 0 - RSS: 2593.47 MB, Heap: 219.95/235.11 MB
2026-01-19T15:17:38+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 0 - Total: 2153.00 MB
2026-01-19T15:17:38+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:38+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:38+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 1 - RSS: 2628.88 MB, Heap: 212.63/235.11 MB
2026-01-19T15:17:38+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 1 - Total: 2153.00 MB
2026-01-19T15:17:38+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:38+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:39+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 2 - RSS: 2670.50 MB, Heap: 214.42/243.11 MB
2026-01-19T15:17:39+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 2 - Total: 2153.00 MB
2026-01-19T15:17:39+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:39+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 3 - RSS: 2690.51 MB, Heap: 213.03/240.06 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 3 - Total: 2153.00 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 4 - RSS: 2709.51 MB, Heap: 209.10/238.68 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 4 - Total: 2153.00 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:41+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 5 - RSS: 2687.46 MB, Heap: 210.02/238.68 MB
2026-01-19T15:17:41+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 5 - Total: 2153.00 MB
2026-01-19T15:17:41+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:41+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:42+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 6 - RSS: 2647.46 MB, Heap: 207.48/238.18 MB
2026-01-19T15:17:42+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 6 - Total: 2153.00 MB
2026-01-19T15:17:42+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:42+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 7 - RSS: 2687.43 MB, Heap: 208.42/238.18 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 7 - Total: 2153.00 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 8 - RSS: 2667.42 MB, Heap: 208.20/237.68 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 8 - Total: 2153.00 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:44+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 9 - RSS: 2689.67 MB, Heap: 211.95/237.68 MB
2026-01-19T15:17:44+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 9 - Total: 2153.00 MB
2026-01-19T15:17:44+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:44+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:45+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 10 - RSS: 2672.50 MB, Heap: 212.04/237.93 MB
2026-01-19T15:17:45+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 10 - Total: 2153.00 MB
2026-01-19T15:17:45+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:45+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:46+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 11 - RSS: 2691.79 MB, Heap: 212.97/237.93 MB
2026-01-19T15:17:46+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 11 - Total: 2153.00 MB
2026-01-19T15:17:46+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:46+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:47+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 12 - RSS: 2672.31 MB, Heap: 207.55/238.18 MB
2026-01-19T15:17:47+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 12 - Total: 2153.00 MB
2026-01-19T15:17:47+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:47+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:48+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 13 - RSS: 2692.31 MB, Heap: 208.69/238.18 MB
2026-01-19T15:17:48+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 13 - Total: 2153.00 MB
2026-01-19T15:17:48+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:48+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:49+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 14 - RSS: 2652.46 MB, Heap: 207.80/238.18 MB
2026-01-19T15:17:49+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 14 - Total: 2153.00 MB
2026-01-19T15:17:49+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:49+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:50+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 15 - RSS: 2692.34 MB, Heap: 209.17/238.18 MB
2026-01-19T15:17:50+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 15 - Total: 2153.00 MB
2026-01-19T15:17:50+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:50+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:51+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 16 - RSS: 2672.25 MB, Heap: 207.72/238.43 MB
2026-01-19T15:17:51+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 16 - Total: 2153.00 MB
2026-01-19T15:17:51+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:51+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:52+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 17 - RSS: 2694.63 MB, Heap: 208.52/238.43 MB
2026-01-19T15:17:52+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 17 - Total: 2153.00 MB
2026-01-19T15:17:52+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:52+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:53+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 18 - RSS: 2653.39 MB, Heap: 209.55/238.43 MB
2026-01-19T15:17:53+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 18 - Total: 2153.00 MB
2026-01-19T15:17:53+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:53+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:54+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 19 - RSS: 2693.16 MB, Heap: 210.79/238.43 MB
2026-01-19T15:17:54+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 19 - Total: 2153.00 MB
2026-01-19T15:17:54+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:54+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:55+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 20 - RSS: 2673.18 MB, Heap: 208.21/237.93 MB
2026-01-19T15:17:55+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 20 - Total: 2153.00 MB
2026-01-19T15:17:55+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:55+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:56+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 21 - RSS: 2694.30 MB, Heap: 210.80/237.93 MB
2026-01-19T15:17:56+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 21 - Total: 2153.00 MB
2026-01-19T15:17:56+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:56+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:57+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 22 - RSS: 2654.76 MB, Heap: 213.88/238.18 MB
2026-01-19T15:17:57+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 22 - Total: 2153.00 MB
2026-01-19T15:17:57+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:57+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:58+0100 <info> gateway.backup.js:167 (Gateway.backup) Gladys backup uploaded with success to Gladys Gateway.
2026-01-19T15:17:58+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory after upload - RSS: 2678.62 MB, Heap: 210.48/238.18 MB
2026-01-19T15:17:58+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory after upload - Total: 2153.00 MB
2026-01-19T15:17:58+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:58+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
38

Thanks a lot for the logs, that helps a lot.

The issue is on the DuckDB side, not on the Node.js side.

I have an idea, I’m going to add a DuckDB CHECKPOINT after the export.

I’ve just restarted a build on the same tag — once it’s ready I’ll give you a heads-up so you can test!

1 Like
39

The build is ready on the same tag!

40 
2026-01-19T15:32:12+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before backup - RSS: 421.17 MB, Heap: 240.86/276.61 MB
2026-01-19T15:32:12+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before backup - Total: 3.50 MB
2026-01-19T15:32:12+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 0.75 MB
2026-01-19T15:32:12+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:32:12+0100 <info> system.isDocker.js:13 (System.isDocker) System.isDocker: This system is running inside Docker.
2026-01-19T15:32:12+0100 <info> gateway.backup.js:79 (Gateway.backup) Gateway backup: Locking SQLite Database
2026-01-19T15:32:12+0100 <info> gateway.backup.js:84 () Backup attempt n°1 : Cleaning backup folder
2026-01-19T15:32:12+0100 <info> gateway.backup.js:88 () Starting Gateway backup in folder /var/lib/gladysassistant/backups/gladys-db-backup-2026-1-19-15-32-12.db
2026-01-19T15:32:12+0100 <info> gateway.backup.js:90 () Gateway backup: Unlocking Database
2026-01-19T15:32:12+0100 <info> gateway.backup.js:96 (Gateway.backup) Gateway backup : SQLite file size is 2mb.
2026-01-19T15:32:12+0100 <info> gateway.backup.js:97 (Gateway.backup) Gateway backup : Backing up DuckDB into a Parquet folder /var/lib/gladysassistant/backups/gladys-db-backup_2026-1-19-15-32-12_parquet_folder
2026-01-19T15:33:07+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory after export database to parquet - RSS: 3457.38 MB, Heap: 209.81/233.86 MB
2026-01-19T15:33:09+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory after export database to parquet - Total: 2154.00 MB
2026-01-19T15:33:09+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2151.25 MB
2026-01-19T15:33:09+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:33:09+0100 <info> gateway.backup.js:108 (Gateway.backup) Gateway backup: Forcing DuckDB checkpoint to release memory
2026-01-19T15:33:11+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory after checkpoint - RSS: 2895.04 MB, Heap: 211.72/236.86 MB
2026-01-19T15:33:11+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory after checkpoint - Total: 2205.25 MB
2026-01-19T15:33:11+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:11+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:11+0100 <info> gateway.backup.js:112 (Gateway.backup) Gateway backup: Compressing backup
2026-01-19T15:33:38+0100 <info> gateway.backup.js:118 (Gateway.backup) Gateway backup: Encrypting backup
2026-01-19T15:33:40+0100 <info> gateway.backup.js:125 (Gateway.backup) Gateway backup: Uploading backup, size of encrypted backup = 444mb. Path = /var/lib/gladysassistant/backups/gladys-db-backup-2026-1-19-15-32-12.tar.gz.enc
2026-01-19T15:33:41+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 0 - RSS: 2650.89 MB, Heap: 216.82/244.06 MB
2026-01-19T15:33:41+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 0 - Total: 2205.25 MB
2026-01-19T15:33:41+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:41+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:44+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 1 - RSS: 2670.77 MB, Heap: 211.36/243.68 MB
2026-01-19T15:33:44+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 1 - Total: 2205.25 MB
2026-01-19T15:33:44+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:44+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:46+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 2 - RSS: 2690.89 MB, Heap: 211.13/243.93 MB
2026-01-19T15:33:46+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 2 - Total: 2205.25 MB
2026-01-19T15:33:46+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:46+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 3 - RSS: 2730.59 MB, Heap: 207.60/244.18 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 3 - Total: 2205.25 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 4 - RSS: 2750.83 MB, Heap: 209.09/244.18 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 4 - Total: 2205.25 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:48+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 5 - RSS: 2730.72 MB, Heap: 207.56/244.18 MB
2026-01-19T15:33:48+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 5 - Total: 2205.25 MB
2026-01-19T15:33:48+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:48+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:49+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 6 - RSS: 2751.00 MB, Heap: 211.59/244.18 MB
2026-01-19T15:33:49+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 6 - Total: 2205.25 MB
2026-01-19T15:33:49+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:49+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 7 - RSS: 2712.23 MB, Heap: 210.08/244.18 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 7 - Total: 2205.25 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 8 - RSS: 2751.04 MB, Heap: 211.45/244.18 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 8 - Total: 2205.25 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:51+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 9 - RSS: 2731.21 MB, Heap: 207.65/244.18 MB
2026-01-19T15:33:51+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 9 - Total: 2205.25 MB
2026-01-19T15:33:51+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:51+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:52+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 10 - RSS: 2751.28 MB, Heap: 209.11/244.18 MB
2026-01-19T15:33:52+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 10 - Total: 2205.25 MB
2026-01-19T15:33:52+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:52+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:53+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 11 - RSS: 2733.05 MB, Heap: 209.46/259.93 MB
2026-01-19T15:33:53+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 11 - Total: 2205.25 MB
2026-01-19T15:33:53+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:53+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:54+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 12 - RSS: 2751.96 MB, Heap: 210.97/259.93 MB
2026-01-19T15:33:54+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 12 - Total: 2205.25 MB
2026-01-19T15:33:54+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:54+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:56+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 13 - RSS: 2773.28 MB, Heap: 213.34/260.18 MB
2026-01-19T15:33:56+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 13 - Total: 2205.25 MB
2026-01-19T15:33:56+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:56+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:57+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 14 - RSS: 2751.95 MB, Heap: 214.58/260.18 MB
2026-01-19T15:33:57+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 14 - Total: 2205.25 MB
2026-01-19T15:33:57+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:57+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:58+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 15 - RSS: 2732.84 MB, Heap: 209.05/260.43 MB
2026-01-19T15:33:58+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 15 - Total: 2205.25 MB
2026-01-19T15:33:58+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:58+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:59+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 16 - RSS: 2751.91 MB, Heap: 210.49/260.43 MB
2026-01-19T15:33:59+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 16 - Total: 2205.25 MB
2026-01-19T15:33:59+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:59+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:34:00+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 17 - RSS: 2732.95 MB, Heap: 209.07/260.43 MB
2026-01-19T15:34:00+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 17 - Total: 2205.25 MB
2026-01-19T15:34:00+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:34:00+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:34:02+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 18 - RSS: 2714.02 MB, Heap: 210.83/259.93 MB
2026-01-19T15:34:02+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 18 - Total: 2205.25 MB
2026-01-19T15:34:02+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:34:02+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:34:03+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 19 - RSS: 2752.82 MB, Heap: 212.62/259.93 MB
2026-01-19T15:34:03+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 19 - Total: 2205.25 MB
2026-01-19T15:34:03+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:34:03+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:34:04+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 20 - RSS: 2732.86 MB, Heap: 207.55/259.93 MB
2026-01-19T15:34:04+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 20 - Total: 2205.25 MB
2026-01-19T15:34:04+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:34:04+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:34:05+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 21 - RSS: 2715.16 MB, Heap: 207.26/260.18 MB
2026-01-19T15:34:05+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 21 - Total: 2205.25 MB
2026-01-19T15:34:05+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:34:05+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 22 - RSS: 2755.16 MB, Heap: 210.93/260.18 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 22 - Total: 2205.25 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:171 (Gateway.backup) Gladys backup uploaded with success to Gladys Gateway.
2026-01-19T15:34:07+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory after upload - RSS: 2759.53 MB, Heap: 211.51/260.43 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory after upload - Total: 2205.25 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
41

Ok that doesn’t help :smiley: I’ll try to create a special connection for the backup