Gladys Assistant 4.45: DuckDB, a revolution in Gladys! ⚡

@pascal_sapin Yes, that’s normal :slight_smile:

@guim31 that’s indeed the case, we now use the raw data instead of precomputed data, and DuckDB does live sampling on the entire dataset so there are fewer « hiccups » when switching between different periods like it used to be!

1 Like

Well, it’s awesome because it looks cool!!! :wink:

2 Likes

I noticed that too, I didn’t expect it to change the shape of the curves so much. And I wouldn’t have guessed it was you, @guim31 :laughing:

2 Likes

So, were there any other nice DB discounts? :grinning_face_with_smiling_eyes:

Normally, everyone should now be up to date!

2 Likes

Bravo Pierre-Gilles, nothing else to say.
Update went very smoothly, no bugs, slowdowns, or anything.

Database size reduced by 100x, no less…

Hats off, artist :clap:

2 Likes

Is it normal that in my tasks I’m still migrating to DuckDB this morning?

for info I restarted Gladys 26 minutes ago and I rebooted my miniPC 1 minute ago

I imagine it’s simply a check at Gladys startup, but those should be instantaneous because there’s nothing left to migrate. The task starts and checks whether any data still needs to be migrated.

Hello. Everything seems to be OK here… Thank you very

Update dropped yesterday at 5:30 PM, migration finished at 1:15 AM.
SQLite database 18.

1 Like

Discussion ongoing here: [RESOLU] Faire des modifications directement dans la BDD.

DBeaver works well. Two of us couldn’t manage to use TablePlus either.

For me, successive Gladys crashes (I’m used to this), but it keeps running anyway.


![Capture d’écran de 2024-08-27 18-29-07|690x387](u

pload://sdIQIrWfG82BRQV4Efd6d5quLM2.png)

![Capture d’écran de 2024-08-27 18-30-40|690x387](upload://10s

Hi @bjm,

What hardware are you running Gladys on again?

Pi 3 + SSD
I had a major crash at the beginning of the year, reinstalled Gladys from a backup (disk image), and since then it crashes approximately every 9 days.

The Pi3 can indeed struggle if your DB is substantial. It seems to me that @GBoulvin had an issue on the Pi4 (during the purge phase, though):

But it worked out ^^

I just remembered that I hadn’t plugged the fan in; it’s done now, but I had to turn Gladys off!

Hi,

For me:
Migration to DuckDB around 5:30 PM last night. Success
Gladys backup started 40 minutes ago. It has just finished.
Size yesterday: 6.79 GB
Size 24 minutes ago: 6.93 GB ??

  • Migration completed: Yes
  • Number of states in SQLite: 38,362,040
  • Number of states in DuckDB: 38,463,404

Excellent !! :heart_eyes: Thanks for the feedback.

Yes, a task runs at each Gladys startup to check whether it needs to migrate or not, that’s normal :slight_smile:

Great :slight_smile:

There must be a hardware issue on your side, it’s really very, very rare for Gladys to crash — it never happened to me in 7 months with quite a substantial setup ^^

On a Pi 3, the RAM is only 1 GB, part of which will be used by the system. It could be an « out of memory » crash. Feel free to create a separate thread on the forum so we can talk about it :slight_smile:

In any case, if you have the opportunity I really recommend you switch to a mini-PC. It’s truly night and day; they’re not very expensive new and really cheap second-hand on leboncoin.

You need to do the SQLite purge + cleanup + reboot.

The entire procedure is described here:

Hello,

Well

1 Like

Okay, thanks @pierre-gilles
I thought, after reading the following message, that the purge could be done afterwards.
https://community.gladysassistant.com/t/gladys-assistant-4-45-duckdb-une-revolution-dans-gladys

1 Like

Hi,

I didn’t really have a big database but it went from 1.6 GB to 860 KB.
That’s not too bad, though.
Thanks for the work

3 Likes