Gladys Assistant 4.83: Faster activity, DuckDB cleanup and 2FA for Gladys Plus

[pierre-gilles]

Hello everyone!

Version 4.83.0 is available!

After the major release 4.82 (activity log + MQTT), this version focuses on reliability, performance, and some very concrete daily improvements, especially for installations with a lot of history, and for Gladys Plus.

---

Activity Page: Instant Display, Even on Large Databases

On an installation with hundreds of millions of states, filtering activity on an infrequent category (openings, buttons…) could freeze the interface for several tens of seconds.

The Activity page now loads the history in progressive time windows: the first events are displayed immediately, and the search continues in the background with a banner of the type « Searching for activity — {month}… ».

On a database of 448 million states, a « Openings » filter that took 20–33 s to respond now displays the first results in ~100 ms.

Thanks to @Terdious for this work!

---

Automatic Cleanup of Orphaned States (DuckDB)

Since the migration of histories to DuckDB, a bug caused the « no longer keep history » purge of a feature, as well as the deletion of devices in some cases, not to clean up states correctly in DuckDB.

Result: « zombie » states (features without history, or already deleted features) could accumulate for years without anyone noticing, until the arrival of the activity log.

This release fixes the purge, and runs once at startup a background job that cleans up orphaned states, gently (in batches, with pauses), to avoid saturating the CPU or blocking the rest of Gladys.

On a test installation, 45 million orphaned states were thus purged. You can follow the progress in Settings → Tasks.

Thanks again to @Terdious!

---

Gladys Plus: 2FA Flow Redesigned

The entire Gladys Plus two-factor authentication flow has been reworked to better handle many small unpleasant cases that had been reported to me (missing configuration, code errors, going back, etc.).

Don’t hesitate to give me your feedback on 2FA. I really want it to be simpler for beginner users, and I am aware that the authentication steps remain a bit complex for a non-technical user.

A next step would be to add recovery codes, to be able to reset the 2FA yourself in case of loss of the phone or the authentication app.

---

Gladys Plus Restoration: No More Temporary Account

When restoring a backup from the registration screen, Gladys previously created a temporary local account with hardcoded credentials. If the flow was interrupted or failed, this account could remain, block registration… and leave an instance in a painful state.

This temporary account has been removed. Restoration works without creating a local user, and instances already « blocked » by an old temporary account are automatically healed at startup.

---

Scenes: Duration in Seconds on State Trigger

On the Device State Change trigger, the option « run after the condition has been valid for… » only offered minutes.

You can now choose seconds or minutes, practical for reacting quickly (ex.: « if movement remains detected for 10 seconds »).

Existing scenes remain in minutes by default.

---

Interface

• Dashboard: widget types are sorted alphabetically according to the interface language (thanks @Will_71)
• Scenes: same for trigger and action lists (thanks @Will_71)
• MQTT: excessive vertical spacing corrected in the device list
• Activity: horizontal scroll of filters improved on Windows
• Scenes: correction of a black gap on multi-line variable chips
• HomeKit: temperature color values out of bounds (ex. LED strips) are now bounded to the max HomeKit (500 mireds), which avoids HAP warnings of the type « characteristic was supplied illegal value »

---

Technical

• DuckDB Migration: switch from the duckdb package (deprecated) to @duckdb/node-api. Same engine, same .duckdb files: no data migration on the user side. Precompiled binaries replace native compilation.
• Automatic publication of the Gladys Plus front-end after a production release

---

Thanks to Contributors

A big thanks to @Terdious and @Will_71 for their contributions to this version, and thanks to the entire community for your feedback and your tests, especially those of you who run Gladys on very large history databases: it’s thanks to you that we can validate these performances in real conditions.

As always, Gladys updates automatically within 24 hours if you use Watchtower, otherwise you can do it in one click in the settings.

Don’t forget to configure Telegram to receive an alert on your phone when Gladys updates!

View the full release notes on GitHub

[prohand]

Thanks for this version

On my end for the purge:

[Terdious]

Oh my my my!!^^ Are you and @Will_71 deleting devices/features every day?

As I was discussing with Pierre-Gilles, I had 20 out of 450 million states ^^ I never delete anything

6 minutes for a complete cleanup?? Wow, is Gladys running on a powerhouse? It takes me over an hour (too many states )

[prohand]

Virtual machine on a Proxmox that’s starting to have quite a few virtual machines

On the virtual machine side:

For the number of states on my side:

[Will_71]

@Terdious

every day with @Will_71

In my defense, I had a lot of trouble with my Ethernet Zigbee SMLIGHT key, which lost half of the devices after a few hours, so I took the opportunity to rename the devices on the Zigbee2mqtt side, even if it meant losing the data. I didn’t have time to rename them in the database.

[Terdious]

Well, the new Beelink SER9 mini PC does the job much better and faster ^^

[mutmut]

Clearly something is happening but what … mystery

[pierre-gilles]

@mutmut I imagine you’re on Gladys Plus, refresh the page and you should see the new translations

[mutmut]

@pierre-gilles

I imagine you’re on Gladys Plus

Not at all, locally like most of the time at home

EDIT: but the refresh works well!

[pierre-gilles]

Gladys is a PWA, so your browser caches the entire frontend. Sometimes you need a good refresh to move to the next version

[guim31]

I just got home and I see this:

Is this normal? Does this mean that if I had any orphaned states, I no longer have them?

[Terdious]

@guim31

Is this normal? Does that mean if I had orphaned states, I don’t have them anymore?

Yes it’s normal!! That means you didn’t have any at all ^^ Like Pierre-Gilles

Well done

[mutmut]

I’m on the side of the big orphans

[Will_71]

@mutmut

I’m on the side of the big orphans

I feel less alone on this side

[Jluc]

The CPU temperature increased by 4°C to perform the orphan purge.

I note that the disk usage rate remained at 16 %.

I thank all contributors for all these bursts of new features and improvements.

As a non-technical user, I observe what’s happening without fully understanding, but the solution is evolving. I manage to take advantage of some new possibilities and I really like it.

[Terdious]

@Jluc

I noticed that the disk occupancy rate remained at 16%

Hi @Jluc, that wouldn’t surprise me given the size of your disk. DuckDB data is very lightweight, and if it’s mostly binary states that have been deleted, it’s almost nothing.
To get an approximate idea (it’s far from accurate, but it’s still an average), you take the size of your DB on the disk and divide it by the number of states you find in the system parameters, and you’ll get the average weight of 1 state ^^

For my part: 9365Mo / 462,000,000 states = 2e-5Mo

If we take this base (really just for demonstration ^^), that gives you 12,045,563 * 2e-5 = 244Mo, or 0.049% of a 500GB disk

It’s still nice to see all these orphans leaving the nest ^^

The impact is much more in terms of time, not helping with performance on the histos. The more data there is to scan, the heavier it becomes to make the queries, even though this data is no longer necessary.