Following the release of 4.82 and discussions with @pierre-gilles about the performance of the new Activity view on heavy installations (at least mine), here’s the summary of PR #2647, tested on my database of 448 million states.
The Problem
On a large database, filtering Activity by a sparse category (e.g., « Openings »: few states, old) forced the server to scan a huge part of the history before responding: the server only responded once its page of 80 states was filled, regardless of the depth to traverse. Result: a frozen spinner for 20 to 50 seconds, sometimes 3 minutes.
We methodically measured all « server-only » leads (progressive windows, anchoring on the last activity — thanks @pierre-gilles for #2642 —, disjoint slices, physical compaction of the table): each improves, none removes the wall — the floor cost is the scan throughput of DuckDB multiplied by the volume to traverse.
The Solution: Let the Client Drive the Search
Server: getDeviceStatesHistory accepts a since bound → one query = a bounded time window, which returns what it contains (even less than 80) and never expands. A bounded window responds in milliseconds thanks to DuckDB zone maps. Without since, behavior strictly unchanged (backward-compatible).
Frontend: The Activity view probes 1 → 2 → 4 → 8 → 16 → 32 months then a final unlimited query, displays each batch as it arrives, with a banner « Searching for activities — March 2025… » while older windows are probed in the background.
The Measurements (448 M states)
Case
Before
After
« Openings » filter (few states, old)
20 to 33 s of frozen spinner
First states in ~100 ms, the page completes in the background
« Buttons » filter (states at 17 months)
22 to 51 s
Immediate banner indicating the scanned month, states as they are found
« All » view / live / chatty sensors
~100 ms
Unchanged (~100 ms)
The total work of the worst case doesn’t change — but it’s done behind an already usable page instead of blocking the first display. This is the principle of the Home Assistant logbook, adapted to the Gladys API.
