Gladys becomes unreachable after a few hours of operation

Configuration
21

docker stats returns info that varies, of course, between calm moments and more intense ones (it looks like a cycle of about 10 seconds) :

image
image1183×187 35.3 KB

or
image
image1183×187 35.3 KB

or
image
image1183×187 35 KB

and htop:

image
image1295×691 135 KB

or
image
image1295×691 136 KB

22

I understand better now: with 1.8 GB of RAM on your Pi, if 600 MB are always used in normal operation, and if in the current state you need more than 1 GB for aggregation (you’ve probably fallen behind), it’s going to choke.

Aggregation works sensor by sensor; you probably have a single sensor that’s responsible for the high RAM usage we see. An analysis of the SQLite database could determine which sensor it is. It may turn out you don’t even use its data and you could delete it ^^ (imagine it’s like the battery history for the last 2 years — we can do without that). If you know how to extract your database, I could run the queries and tell you which sensor is causing the problem. Otherwise you can run the queries yourself if you know how to do it too :slight_smile:

If that seems complicated, you can already go through each device in the Gladys-Zigbee2mqtt integration and check that you only keep the states you need. If you uncheck a feature, Gladys will automatically purge the historical states.

1 Like
23

Ok, thanks for the analysis. And it forces me to do what I’ve been putting off for a while: review the configuration of each Zigbee device and each MQTT device to remove as much history as possible. I’ll do that calmly first.

And also look into switching to the mini-PC whose merits you regularly praise :wink:

1 Like
24

The mini PC, honestly it’s worth it
 after you can find good second-hand deals on Le Bon Coin and a Raspberry Pi 4 still resells pretty well.

2 Likes
25

@pierre-gilles The error « SyntaxError: Value expected (char 1) Â» occurs because of a scene that contained this block « Continue only if Â»:

image
image776×265 18.4 KB

1 Like
26

Thanks for the feedback, validation is missing on this scene action! Can you create a GitHub issue on the Gladys repo so we have a record?

27

VoilĂ , c’est fait : Some validation is missing in the scene action "Continuer seulement si" · Issue #2106 · GladysAssistant/Gladys · GitHub

1 Like
28

Quick update on my cleanup:

  • GladysPlus backup of 5.2 GB before cleanup
  • Review of my 56 Zigbee devices. For most, I had left the default configuration which logs all indicators
  • In total, I went from 270 logged indicators to only 80, a
1 Like
29

So I switched from the rPi to a mini-PC (with 8GB of RAM). And no more aggregation issues :wink:

30

We’ll see, once we’ve moved to DuckDB, we won’t have the same storage issues anymore, so the current

31

Hi,
Personal feedback regarding my PIR sensors, which by default had retained the history of all motion for the past 6 months
 (I currently have 6 PIRs in the main rooms, potentially generating motion every 10 seconds
).
Perhaps sensors should only be logged by default for certain types (temperature, humidity
)?
Have a nice day,
Jean

32

That could be an option — after that I think we’ll see what it looks like with DuckDB because this kind of data really won’t take up much space anymore (the 0/1s will be highly compressed by DuckDB).

Especially since we’ll also soon have historical motion sensor visualization on the dashboard, it’ll be handy to have that data :wink:

2 Likes