I have the 2 consumption calculations that were not progressing very fast and looking at the CPU curves (4 vCPU), it did not exceed 10%.
Are we on single-thread for calculations or am I completely wrong?
The crash is at 22:17 on the graph, the white corresponds to unretrieved information, Gladys was working very well.
On the disk side, I have ups and downs and since the crash I have remained very high!
After investigating, it’s my Proxmox script that sees the RAM is maxed out (5.9GB/6GB) during migration and then restarts the Gladys Docker container.
I’m going to disable that.
But how much RAM is needed to migrate a device without migrating its history?
I get the impression it uses a lot of memory.
I don’t have a lot of scenes/dashboards where I use the anemometer for example (that’s what I was trying to migrate.
My duckDB is almost 10GB, does that matter?
Anyway, yesterday I migrated the device without any history and it caused the RAM to spike until it crashed (6GB) and then manual restart due to OOM.
Today I increased my LXC from 6 to 8GB of RAM, launched the migration without history: super fast!
Oh indeed, if your database is 10 GB and there are many states to migrate, then yes it can take a lot of time and a lot of RAM.
Do you have the possibility to temporarily allocate more RAM just for the duration of the migration?
Regarding the memory that “doesn’t free up” afterward: this is normal and it’s not a leak. DuckDB keeps its pages in cache up to its memory limit and the system doesn’t immediately release it. It is reused by Gladys and freed under pressure.
And I advise you to disable automatic restart on RAM threshold during this kind of operation: restarting Gladys in the middle of a migration leaves the work half done (it’s replayable, but it’s better to avoid it).
already done but I’ll have to go to 10 or 12GB because I’ve been migrating for over an hour and the 8GB is full and Gladys is crashing, the dashboard is no longer refreshing
ok I’ll monitor then to see what happens over time.
@mutmut, No issues on my end with a configuration almost identical to yours.
I migrated my Netatmo thermostat without any problems, no slowdowns, and it was very quick. My database is 6 GB
I managed to upgrade my LXC to 12GB live without rebooting, and the migration of my external station (with everything to migrate) took 1 hour and 26 minutes, with significant slowdowns in consumption calculations during the migration.
