RAM problem - Gladys no longer responds

Configuration
1

Hello,

For a few days now, I’ve been having a lot of trouble with my Gladys instance, since approximately the last update. Probably a coincidence…

Here’s what’s happening: my Gladys takes longer and longer to load pages until it reaches saturation. It eventually stops responding to pings, and I have to physically restart it.

I have a Raspberry PI 4 with an SD card, and I don’t have any issues with the other containers (except when the machine stops responding to pings, of course).

Looking at the stats a bit, I notice that my memory is swelling very quickly:

image
image940×159 6.32 KB

If I restart by stopping and starting the Gladys container, the memory goes back to 0 and continues to swell continuously until saturation.

As for the Gladys logs, the last few lines are as follows:

Forum post limited to 33,000 characters
c6e3b76b2ee4','hourly','ba4e30b3-bb76-4e8b-a61a-feaa42c53cf7',6176.74,'2023-03-03 11:50:25.428 +00:00','2023-03-03 11:50:25.428 +00:00'),('12a018e2-de2b-4064-bc43-37dad72d4418','hourly','ba4e30b3-bb76-4e8b-a61a-feaa42c53cf7',6176.74,'2023-03-03 11:50:49.622 +00:00','2023-03-03 11:50:49.622 +00:00'),('d028d2a3-ae03-4894-8f56-1a75b6bc9f7b','hourly','ba4e30b3-bb76-4e8b-a61a-feaa42c53cf7',6176.75,'2023-03-03 11:50:49.900 +00:00','2023-03-03 11:50:49.900 +00:00'),('00b54221-d81c-4ba1-95ca-f435d5c62f64','hourly','ba4e30b3-bb76-4e8b-a61a-feaa42c53cf7',6176.75,'2023-03-03 11:50:53.704 +00:00','2023-03-03 11:50:53.704 +00:00'),('287dac16-70bf-4d72-b44c-e3fefba1b122','hourly','ba4e30b3-bb76-4e8b-a61a-feaa42c53cf7',6176.77,'2023-03-03 11:50:56.318 +00:00','2023-03-03 11:50:56.318 +00:00'),('93b65da7-7ea2-49e6-a797-8c715b169425','hourly','ba4e30b3-bb76-4e8b-a61a-feaa42c53cf7',6176.77,'2023-03-03 11:51:09.528 +00:00','2023-03-03 11:51:09.528 +00:00'),('a68474c1-0118-4580-88a8-a1a0298dc5db','hourly','ba4e30b3-bb76-4e8b-a61a-feaa42c53cf7',6176.79,'2023-03-03 11:51:10.957 +00:00','2023-03-03 11:51:10.957 +00:00'),('13a8eb25-0bb3-44c7-a541-7fd7ccd66c82','hourly','ba4e30b3-bb76-4e8b-a61a-feaa42c53cf7',6176.79,'2023"..
2

By continuing to browse and search the forum, I get the impression that it looks a bit like this:

As I recall, I did the following recently:

  • Added MQTT items to display prices relative to my energy consumption/production
  • Modified an MQTT item because the topic name was incorrect
  • Deleted 2 charts because I thought they were the cause of the slowness
  • Performed a cleanup (impossible to know if it completed correctly since 24h later, no more access
3

I think the problem comes from my DB being over 14GB. I made the bad decision to save everything.
You learn by making mistakes.

However, since the system is completely down, how can I modify the DB manually to delete the data in t_device_feature_state to start?
I exported the DB and used DB Browser on Windows but is it possible to stop the container and then manually perform the operation on the DB?
I’m a novice with SQLite and even more so with containers :confused:

4

Can someone explain how to run SQL commands directly in Docker? :frowning:
The sqlite3 command doesn’t exist. Do I need to install the package?

5

Sorry @Kevin for the silence… Unfortunately I have no idea how to do it :confused:

6

Hello,

Indeed, sqlite3 is installed only in the gladys container but performing the operations while the container is running is a bad idea.

The best (and easiest if you’re not familiar with the command line) is to:

  • stop the Gladys container
    docker stop gladys
  • make a backup of the database
    cp /var/lib/gladysassistant/gladys-production.db /var/lib/gladysassistant/gladys-production.db.backup
  • copy the file /var/lib/gladysassistant/gladys-production.db to another machine
  • use an editor like DBBrowser
  • make your changes (be sure to Save changes)
  • copy the new database back to /var/lib/gladysassistant/
  • start Gladys
    docker start gladys
7

Hello,

Thank you for your detailed reply :slight_smile:

I have no problem with the command line, I was a DEV in C# .NET but I was more used to working with stored procedures, transactions and other things on SQL Server instances with SSMS… And having no experience with Docker or SQLite, I was a bit lost and needed a very small getting-started guide :smiling_face:

I’ve already recovered the DB after stopping the Gladys container. And I’m doing some manipulations via DB Browser. However, I had two files (***.DB-SHM, ***.DB-WAL) in addition to the DB (cache, queue , … ?) and I wonder if I need to force a merge or something? Or can I simply ignore these files without corrupting my DB if I later put back my modified DB?

Thanks again for your help @cicoub13 ! :slightly_smiling_face:

@guim31 : No worries, thanks anyway for replying :slight_smile:

8

No worries about these files, you can ignore them. When Gladys restarts, the SHM will start from scratch. The WAL contains the rollback procedures in case of an error. But here, we shut down the database connection cleanly.

9

Ok, perfect. Thank you for this additional information. I will proceed accordingly.

10

Uh no, I don’t agree

Normally if you properly stop Gladys, these files should disappear (they only exist when a SQLite program is accessing the DB). You need to make sure that’s the case before copying the DB file, or else copy the 3 files.

If you ever copy a new SQLite DB next to old db-shm / db-wal files, you risk database corruption.

11

For my part, it worked by ignoring those files. Was it luck? I don’t know. Afterwards, Gladys was properly shut down, but given the 14GB, I suppose it wasn’t able to do what was necessary with the other two files.