Plantage container Gladys / mémoire pleine la nuit

Checconio · Janvier 11, 2026, 8:25

Parce que je sors un graphique temps réel de la puissance VA ^^

Mais la réduction à un payload ne semble pas avoir changé le souci de mémoire.

Je vais continuer à creuser.

pierre-gilles · Janvier 12, 2026, 9:33

Est-ce que tu pourrais quand même tester de passer à une fréquence plus faible, juste pour voir si c’est ça le souci ? Toutes les secondes, ça me paraît très agressif ! ^^

Il se passe beaucoup de choses quand une valeur de capteur est reçue, notamment Gladys vérifie qu’aucune scène ne doit se déclencher. Là, vu le nombre de valeurs que tu reçois pour chaque payload toutes les secondes, ça fait lourd pour Gladys je pense.

_Will_71 · Janvier 12, 2026, 11:26

J’ai pour ma part eu aussi ce souci dans la nuit de samedi à dimanche à 3h du matin.
J’ai eu un pic de consommation de la ram jusqu’au max et ensuite cela a planté.
Je n’ai pas plus de détails mais si cela recommence j’investigerai plus sur le sujet.

Checconio · Janvier 12, 2026, 5:57

Je viens de passer à 60 secondes pour voir si c’est mieux

Checconio · Janvier 14, 2026, 8:55

Le souci est toujours présent.

Ça ne vient donc pas des requêtes que mon module Linky effectue.
Je vais voir comment je peux creuser un peu plus au niveau des processus.

pierre-gilles · Janvier 15, 2026, 8:16

Il faut voir si tu n’as pas un autre appareil très agressif ?

Checconio · Janvier 15, 2026, 8:32

J’ai effectivement des détecteurs de mouvement qui envoient des rapports assez fréquents.

Je vais voir pour limiter tout ce qui apparaît souvent dans les logs.

Checconio · Janvier 18, 2026, 4:44

Bon, j’ai creusé plus en détail dans la mémoire du container.

Après quelques jours de reporting, j’ai pu identifier que la mémoire est saturée par DuckDB ou du moins une requête vers DuckDB.
J’ai lancé un script d’analyse de chaque requête Node.js DuckDB afin d’identifier quelle requête vers DuckDB pose problème.
Sûrement lié au fait que ma base est assez conséquente.

J’ai fait un gros nettoyage dans les métriques qui n’avaient pas besoin d’historique afin d’alléger la base.
Depuis, ça m’a l’air beaucoup plus stable, mais du coup je risque de ne pas savoir quel était le problème avant ^^

Si la montée en mémoire ne se reproduit pas, je vais restaurer ma base avant nettoyage pour reproduire le souci.

pierre-gilles · Janvier 19, 2026, 7:57

C’est probablement la requête d’affichage graphique, si jamais à un moment tu as ta base de donnée avec le souci, hésite pas à l’exporter et à me l’envoyer pour que je fasse des tests dessus !

Checconio · Janvier 19, 2026, 10:51

Oui, j’attends que le problème se reproduise concernant le gros pic de mémoire.

J’ai aussi pu remarquer que lors d’un backup Gateway, la mémoire augmente (ce qui est tout à fait normal), mais celle-ci n’est pas libérée après la fin du backup.

Exemple backup de ce matin:

2026-01-19T03:51:19+0100 <info> gateway.checkIfBackupNeeded.js:44 (Timeout._onTimeout) Starting backup!
2026-01-19T03:51:19+0100 <info> system.isDocker.js:13 (System.isDocker) System.isDocker: This system is running inside Docker.
2026-01-19T03:51:19+0100 <info> gateway.backup.js:55 (Gateway.backup) Gateway backup: Locking SQLite Database
2026-01-19T03:51:19+0100 <info> gateway.backup.js:60 () Backup attempt n°1 : Cleaning backup folder
2026-01-19T03:51:19+0100 <info> gateway.backup.js:64 () Starting Gateway backup in folder /var/lib/gladysassistant/backups/gladys-db-backup-2026-1-19-3-51-19.db
2026-01-19T03:51:19+0100 <info> gateway.backup.js:66 () Gateway backup: Unlocking Database
2026-01-19T03:51:19+0100 <info> gateway.backup.js:72 (Gateway.backup) Gateway backup : SQLite file size is 2mb.
2026-01-19T03:51:19+0100 <info> gateway.backup.js:73 (Gateway.backup) Gateway backup : Backing up DuckDB into a Parquet folder /var/lib/gladysassistant/backups/gladys-db-backup_2026-1-19-3-51-19_parquet_folder
2026-01-19T03:52:14+0100 <info> gateway.backup.js:82 (Gateway.backup) Gateway backup: Compressing backup
2026-01-19T03:52:42+0100 <info> gateway.backup.js:88 (Gateway.backup) Gateway backup: Encrypting backup
2026-01-19T03:52:43+0100 <info> gateway.backup.js:95 (Gateway.backup) Gateway backup: Uploading backup, size of encrypted backup = 443mb. Path = /var/lib/gladysassistant/backups/gladys-db-backup-2026-1-19-3-51-19.tar.gz.enc
2026-01-19T03:53:04+0100 <info> gateway.backup.js:139 (Gateway.backup) Gladys backup uploaded with success to Gladys Gateway.

pierre-gilles · Janvier 19, 2026, 10:54

Tu es sûr que tu mesure la bonne chose ? Comment mesures-tu l’utilisation mémoire ?

Checconio · Janvier 19, 2026, 11:00

Là, je ne suis pas rentré dans le détail de la mémoire, j’ai vu ça ce matin par hasard.

C’est la mémoire globale, mais l’augmentation est pile poil au moment du backup.

Ça commence à 3H51, exactement au lancement du backup SQLite et DuckDB dans les logs.

pierre-gilles · Janvier 19, 2026, 12:21

Oui oui, l’augmentation se produit bien au moment du backup.

Je voulais juste comprendre comment tu mesures concrètement l’utilisation de la RAM ?

Selon la méthode utilisée, tu ne verras pas forcément la RAM se « libérer » après le backup, et c’est tout à fait normal sous Linux.

Certains outils montre la RAM totale utilisée en incluant le cache et les buffers, ce qui peut donner l’impression qu’il n’y a pas de RAM libérée, alors que la RAM est bien libérée et réutilisable immédiatement.

Exemple chez moi :

Ce qui est important, c’est « available », pas « free » !

Mais c’est aussi possible qu’il y ait un souci de libération de mémoire, c’est pour ça que le détail technique aiderait beaucoup

Checconio · Janvier 19, 2026, 12:53

Ha oui je comprends, la commande qui est faite pour récupérer mon graphique :

free | grep 'Mem' | awk '{print ($3/$2)*100}'

Du coup, je viens de faire le test

Avant backup :

5 min Après backup

pierre-gilles · Janvier 19, 2026, 1:47

J’ai fait des tests en local chez moi avec de grosses bases de données DuckDB, et je ne trouve rien qui me saute aux yeux.

Après, la réalité du terrain peut être différente, donc j’ai fait un build spécial qui rajoute des logs à la version de prod, pour voir d’où vient l’utilisation de la RAM

Avec ce build, tu verras l’utilisation de la RAM de Node.js + DuckDB avant le backup, pendant le backup, et après le backup !

Le build est disponible sur le tag ci-dessous, est-ce que tu peux tester ?

Je n’ai fais qu’un build x64, car de mémoire tu es sur mini-PC?

(Pas de build arm64 pour gagner du temps)

gladysassistant/gladys:add-memory-logs-during-backup

Si tu veux que le test soit le plus représentatif possible, ne fais rien sur Gladys avant, et pendant le test :

Pas d’ouverture du tableau de bord
Éviter que le test soit à 00:00 ou 00:30 pour ne pas passer sur le calcul des consommations énergétiques

Checconio · Janvier 19, 2026, 2:19

Parfait, je viens de faire le test, je n’ai ouvert aucun dashboard après le démarrage du container.

2026-01-19T15:16:11+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before backup - RSS: 375.19 MB, Heap: 214.96/234.36 MB
2026-01-19T15:16:11+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before backup - Total: 3.50 MB
2026-01-19T15:16:11+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 0.75 MB
2026-01-19T15:16:11+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:16:11+0100 <info> system.isDocker.js:13 (System.isDocker) System.isDocker: This system is running inside Docker.
2026-01-19T15:16:11+0100 <info> gateway.backup.js:79 (Gateway.backup) Gateway backup: Locking SQLite Database
2026-01-19T15:16:11+0100 <info> gateway.backup.js:84 () Backup attempt n°1 : Cleaning backup folder
2026-01-19T15:16:11+0100 <info> gateway.backup.js:88 () Starting Gateway backup in folder /var/lib/gladysassistant/backups/gladys-db-backup-2026-1-19-15-16-11.db
2026-01-19T15:16:11+0100 <info> gateway.backup.js:90 () Gateway backup: Unlocking Database
2026-01-19T15:16:11+0100 <info> gateway.backup.js:96 (Gateway.backup) Gateway backup : SQLite file size is 2mb.
2026-01-19T15:16:11+0100 <info> gateway.backup.js:97 (Gateway.backup) Gateway backup : Backing up DuckDB into a Parquet folder /var/lib/gladysassistant/backups/gladys-db-backup_2026-1-19-15-16-11_parquet_folder
2026-01-19T15:17:06+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory after export database to parquet - RSS: 3574.57 MB, Heap: 212.67/227.86 MB
2026-01-19T15:17:08+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory after export database to parquet - Total: 2153.00 MB
2026-01-19T15:17:08+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:08+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:08+0100 <info> gateway.backup.js:108 (Gateway.backup) Gateway backup: Compressing backup
2026-01-19T15:17:36+0100 <info> gateway.backup.js:114 (Gateway.backup) Gateway backup: Encrypting backup
2026-01-19T15:17:37+0100 <info> gateway.backup.js:121 (Gateway.backup) Gateway backup: Uploading backup, size of encrypted backup = 444mb. Path = /var/lib/gladysassistant/backups/gladys-db-backup-2026-1-19-15-16-11.tar.gz.enc
2026-01-19T15:17:38+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 0 - RSS: 2593.47 MB, Heap: 219.95/235.11 MB
2026-01-19T15:17:38+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 0 - Total: 2153.00 MB
2026-01-19T15:17:38+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:38+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:38+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 1 - RSS: 2628.88 MB, Heap: 212.63/235.11 MB
2026-01-19T15:17:38+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 1 - Total: 2153.00 MB
2026-01-19T15:17:38+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:38+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:39+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 2 - RSS: 2670.50 MB, Heap: 214.42/243.11 MB
2026-01-19T15:17:39+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 2 - Total: 2153.00 MB
2026-01-19T15:17:39+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:39+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 3 - RSS: 2690.51 MB, Heap: 213.03/240.06 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 3 - Total: 2153.00 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 4 - RSS: 2709.51 MB, Heap: 209.10/238.68 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 4 - Total: 2153.00 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:40+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:41+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 5 - RSS: 2687.46 MB, Heap: 210.02/238.68 MB
2026-01-19T15:17:41+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 5 - Total: 2153.00 MB
2026-01-19T15:17:41+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:41+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:42+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 6 - RSS: 2647.46 MB, Heap: 207.48/238.18 MB
2026-01-19T15:17:42+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 6 - Total: 2153.00 MB
2026-01-19T15:17:42+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:42+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 7 - RSS: 2687.43 MB, Heap: 208.42/238.18 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 7 - Total: 2153.00 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 8 - RSS: 2667.42 MB, Heap: 208.20/237.68 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 8 - Total: 2153.00 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:43+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:44+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 9 - RSS: 2689.67 MB, Heap: 211.95/237.68 MB
2026-01-19T15:17:44+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 9 - Total: 2153.00 MB
2026-01-19T15:17:44+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:44+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:45+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 10 - RSS: 2672.50 MB, Heap: 212.04/237.93 MB
2026-01-19T15:17:45+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 10 - Total: 2153.00 MB
2026-01-19T15:17:45+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:45+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:46+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 11 - RSS: 2691.79 MB, Heap: 212.97/237.93 MB
2026-01-19T15:17:46+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 11 - Total: 2153.00 MB
2026-01-19T15:17:46+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:46+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:47+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 12 - RSS: 2672.31 MB, Heap: 207.55/238.18 MB
2026-01-19T15:17:47+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 12 - Total: 2153.00 MB
2026-01-19T15:17:47+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:47+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:48+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 13 - RSS: 2692.31 MB, Heap: 208.69/238.18 MB
2026-01-19T15:17:48+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 13 - Total: 2153.00 MB
2026-01-19T15:17:48+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:48+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:49+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 14 - RSS: 2652.46 MB, Heap: 207.80/238.18 MB
2026-01-19T15:17:49+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 14 - Total: 2153.00 MB
2026-01-19T15:17:49+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:49+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:50+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 15 - RSS: 2692.34 MB, Heap: 209.17/238.18 MB
2026-01-19T15:17:50+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 15 - Total: 2153.00 MB
2026-01-19T15:17:50+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:50+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:51+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 16 - RSS: 2672.25 MB, Heap: 207.72/238.43 MB
2026-01-19T15:17:51+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 16 - Total: 2153.00 MB
2026-01-19T15:17:51+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:51+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:52+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 17 - RSS: 2694.63 MB, Heap: 208.52/238.43 MB
2026-01-19T15:17:52+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 17 - Total: 2153.00 MB
2026-01-19T15:17:52+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:52+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:53+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 18 - RSS: 2653.39 MB, Heap: 209.55/238.43 MB
2026-01-19T15:17:53+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 18 - Total: 2153.00 MB
2026-01-19T15:17:53+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:53+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:54+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 19 - RSS: 2693.16 MB, Heap: 210.79/238.43 MB
2026-01-19T15:17:54+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 19 - Total: 2153.00 MB
2026-01-19T15:17:54+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:54+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:55+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 20 - RSS: 2673.18 MB, Heap: 208.21/237.93 MB
2026-01-19T15:17:55+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 20 - Total: 2153.00 MB
2026-01-19T15:17:55+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:55+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:56+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 21 - RSS: 2694.30 MB, Heap: 210.80/237.93 MB
2026-01-19T15:17:56+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 21 - Total: 2153.00 MB
2026-01-19T15:17:56+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:56+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:57+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 22 - RSS: 2654.76 MB, Heap: 213.88/238.18 MB
2026-01-19T15:17:57+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 22 - Total: 2153.00 MB
2026-01-19T15:17:57+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:57+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:17:58+0100 <info> gateway.backup.js:167 (Gateway.backup) Gladys backup uploaded with success to Gladys Gateway.
2026-01-19T15:17:58+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory after upload - RSS: 2678.62 MB, Heap: 210.48/238.18 MB
2026-01-19T15:17:58+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory after upload - Total: 2153.00 MB
2026-01-19T15:17:58+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2150.25 MB
2026-01-19T15:17:58+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB

pierre-gilles · Janvier 19, 2026, 2:23

Merci beaucoup pour les logs, ça aide beaucoup.

Le souci est côté DuckDB, pas côté Node.js déjà.

J’ai une idée, je vais rajouter un CHECKPOINT DuckDB après l’export.

Je viens de relancer un build sur le même tag, une fois que c’est prêt je te fais signe pour que tu puisse tester !

pierre-gilles · Janvier 19, 2026, 2:27

Le build est prêt sur le même tag !

Checconio · Janvier 19, 2026, 2:45

2026-01-19T15:32:12+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before backup - RSS: 421.17 MB, Heap: 240.86/276.61 MB
2026-01-19T15:32:12+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before backup - Total: 3.50 MB
2026-01-19T15:32:12+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 0.75 MB
2026-01-19T15:32:12+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:32:12+0100 <info> system.isDocker.js:13 (System.isDocker) System.isDocker: This system is running inside Docker.
2026-01-19T15:32:12+0100 <info> gateway.backup.js:79 (Gateway.backup) Gateway backup: Locking SQLite Database
2026-01-19T15:32:12+0100 <info> gateway.backup.js:84 () Backup attempt n°1 : Cleaning backup folder
2026-01-19T15:32:12+0100 <info> gateway.backup.js:88 () Starting Gateway backup in folder /var/lib/gladysassistant/backups/gladys-db-backup-2026-1-19-15-32-12.db
2026-01-19T15:32:12+0100 <info> gateway.backup.js:90 () Gateway backup: Unlocking Database
2026-01-19T15:32:12+0100 <info> gateway.backup.js:96 (Gateway.backup) Gateway backup : SQLite file size is 2mb.
2026-01-19T15:32:12+0100 <info> gateway.backup.js:97 (Gateway.backup) Gateway backup : Backing up DuckDB into a Parquet folder /var/lib/gladysassistant/backups/gladys-db-backup_2026-1-19-15-32-12_parquet_folder
2026-01-19T15:33:07+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory after export database to parquet - RSS: 3457.38 MB, Heap: 209.81/233.86 MB
2026-01-19T15:33:09+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory after export database to parquet - Total: 2154.00 MB
2026-01-19T15:33:09+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2151.25 MB
2026-01-19T15:33:09+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 2.75 MB
2026-01-19T15:33:09+0100 <info> gateway.backup.js:108 (Gateway.backup) Gateway backup: Forcing DuckDB checkpoint to release memory
2026-01-19T15:33:11+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory after checkpoint - RSS: 2895.04 MB, Heap: 211.72/236.86 MB
2026-01-19T15:33:11+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory after checkpoint - Total: 2205.25 MB
2026-01-19T15:33:11+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:11+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:11+0100 <info> gateway.backup.js:112 (Gateway.backup) Gateway backup: Compressing backup
2026-01-19T15:33:38+0100 <info> gateway.backup.js:118 (Gateway.backup) Gateway backup: Encrypting backup
2026-01-19T15:33:40+0100 <info> gateway.backup.js:125 (Gateway.backup) Gateway backup: Uploading backup, size of encrypted backup = 444mb. Path = /var/lib/gladysassistant/backups/gladys-db-backup-2026-1-19-15-32-12.tar.gz.enc
2026-01-19T15:33:41+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 0 - RSS: 2650.89 MB, Heap: 216.82/244.06 MB
2026-01-19T15:33:41+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 0 - Total: 2205.25 MB
2026-01-19T15:33:41+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:41+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:44+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 1 - RSS: 2670.77 MB, Heap: 211.36/243.68 MB
2026-01-19T15:33:44+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 1 - Total: 2205.25 MB
2026-01-19T15:33:44+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:44+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:46+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 2 - RSS: 2690.89 MB, Heap: 211.13/243.93 MB
2026-01-19T15:33:46+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 2 - Total: 2205.25 MB
2026-01-19T15:33:46+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:46+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 3 - RSS: 2730.59 MB, Heap: 207.60/244.18 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 3 - Total: 2205.25 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 4 - RSS: 2750.83 MB, Heap: 209.09/244.18 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 4 - Total: 2205.25 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:47+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:48+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 5 - RSS: 2730.72 MB, Heap: 207.56/244.18 MB
2026-01-19T15:33:48+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 5 - Total: 2205.25 MB
2026-01-19T15:33:48+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:48+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:49+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 6 - RSS: 2751.00 MB, Heap: 211.59/244.18 MB
2026-01-19T15:33:49+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 6 - Total: 2205.25 MB
2026-01-19T15:33:49+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:49+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 7 - RSS: 2712.23 MB, Heap: 210.08/244.18 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 7 - Total: 2205.25 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 8 - RSS: 2751.04 MB, Heap: 211.45/244.18 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 8 - Total: 2205.25 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:50+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:51+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 9 - RSS: 2731.21 MB, Heap: 207.65/244.18 MB
2026-01-19T15:33:51+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 9 - Total: 2205.25 MB
2026-01-19T15:33:51+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:51+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:52+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 10 - RSS: 2751.28 MB, Heap: 209.11/244.18 MB
2026-01-19T15:33:52+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 10 - Total: 2205.25 MB
2026-01-19T15:33:52+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:52+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:53+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 11 - RSS: 2733.05 MB, Heap: 209.46/259.93 MB
2026-01-19T15:33:53+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 11 - Total: 2205.25 MB
2026-01-19T15:33:53+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:53+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:54+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 12 - RSS: 2751.96 MB, Heap: 210.97/259.93 MB
2026-01-19T15:33:54+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 12 - Total: 2205.25 MB
2026-01-19T15:33:54+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:54+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:56+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 13 - RSS: 2773.28 MB, Heap: 213.34/260.18 MB
2026-01-19T15:33:56+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 13 - Total: 2205.25 MB
2026-01-19T15:33:56+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:56+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:57+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 14 - RSS: 2751.95 MB, Heap: 214.58/260.18 MB
2026-01-19T15:33:57+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 14 - Total: 2205.25 MB
2026-01-19T15:33:57+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:57+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:58+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 15 - RSS: 2732.84 MB, Heap: 209.05/260.43 MB
2026-01-19T15:33:58+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 15 - Total: 2205.25 MB
2026-01-19T15:33:58+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:58+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:33:59+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 16 - RSS: 2751.91 MB, Heap: 210.49/260.43 MB
2026-01-19T15:33:59+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 16 - Total: 2205.25 MB
2026-01-19T15:33:59+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:33:59+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:34:00+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 17 - RSS: 2732.95 MB, Heap: 209.07/260.43 MB
2026-01-19T15:34:00+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 17 - Total: 2205.25 MB
2026-01-19T15:34:00+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:34:00+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:34:02+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 18 - RSS: 2714.02 MB, Heap: 210.83/259.93 MB
2026-01-19T15:34:02+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 18 - Total: 2205.25 MB
2026-01-19T15:34:02+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:34:02+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:34:03+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 19 - RSS: 2752.82 MB, Heap: 212.62/259.93 MB
2026-01-19T15:34:03+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 19 - Total: 2205.25 MB
2026-01-19T15:34:03+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:34:03+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:34:04+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 20 - RSS: 2732.86 MB, Heap: 207.55/259.93 MB
2026-01-19T15:34:04+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 20 - Total: 2205.25 MB
2026-01-19T15:34:04+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:34:04+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:34:05+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 21 - RSS: 2715.16 MB, Heap: 207.26/260.18 MB
2026-01-19T15:34:05+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 21 - Total: 2205.25 MB
2026-01-19T15:34:05+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:34:05+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory before upload chunk 22 - RSS: 2755.16 MB, Heap: 210.93/260.18 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory before upload chunk 22 - Total: 2205.25 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:171 (Gateway.backup) Gladys backup uploaded with success to Gladys Gateway.
2026-01-19T15:34:07+0100 <info> gateway.backup.js:35 (logMemoryUsage) JS memory after upload - RSS: 2759.53 MB, Heap: 211.51/260.43 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:41 (logMemoryUsage) DuckDB memory after upload - Total: 2205.25 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:46 ()   - BASE_TABLE: 2203.75 MB
2026-01-19T15:34:07+0100 <info> gateway.backup.js:46 ()   - IN_MEMORY_TABLE: 1.50 MB

pierre-gilles · Janvier 19, 2026, 2:48

Ok ça n’aide pas Je vais essayer de créer une connection spéciale pour le backup

Sujet		Réponses	Vues
[RESOLU] Lenteur parfois sur dashboard Gladys Plus Configuration	48	1921	Février 23, 2021
Help! Problème avec l'agrégation? Configuration	90	3546	Mars 29, 2022
Bug affichage box - timeout réponse instance locale via Gladys Plus Configuration	34	937	Septembre 7, 2022
Reflexion autour de l'API MQTT Développement	17	2936	Mai 4, 2020
Zigbee2MQTT plante 2 fois en 2 heures Configuration	7	836	Juillet 29, 2022

Plantage container Gladys / mémoire pleine la nuit

Sujets connexes