Gladys Assistant 4.66: Energy monitoring and Zigbee2mqtt 2.7.1!

Apparently my teleinformation (MQTT) has indexes while the rest has a consumed power (MQTT and Z2M).
IMG_8780
IMG_8779

The daily information

Strange, I can see them on my side, can you confirm you have:

  • go to the mqtt page
  • Then press F12 and go to the « Network » tab
  • Then type « Borne VE » in the search to show only that one
  • Select the last « device?order_dir=asc\u0026search=… » and expand the device’s features
  • Search for the feature « Energy consumed » (type « energy ») and note its id.
  • Then search for the feature of type « thirty-minutes-consumption » and look at its « energy_parent_id »; it must match the previous id. Then note its id
  • Similarly, search for the « thirty-minutes-consumption-cost » feature and look at its « energy_parent_id »; it must correspond to the previous id, this time of the feature of type « thirty-minutes-consumption ».
    image

If these conditions are not met it won’t work. But as a result that would mean that in the « My devices » tree of the « Energy monitoring » they are not properly positioned. I put this limitation to:

  • on the one hand avoid an overly long list
  • on the other hand avoid running a calculation on the wrong tree structure.

Did you create these features manually before Pierre-Gilles’ update or automatically with the update?

I had created all those features well before, and since I’m on my phone for now, I’ll look at them in detail later on the web.

I just checked and the child/parent are correct.
However, the parent_id of the EV charging station (borne VE) does not match the id of my main meter even though my « Suivi d’énergie » tree is correct and I don’t know if that’s normal:

After checking

Hi @mutmut,

Sorry, a family emergency prevented me from continuing.

I’ve just relaunched an image build with the changes mentioned above
AND a fix that, I hope, will resolve the issue with the features you couldn’t see!!

  • If you can retest.

  • Also, do a test of restarting Gladys during a recalculation to tell me what you think of the logs in the tasks (for example you wait 5 minutes before shutting down and on reconnecting you go to the tasks, you should see where it was - which day - it was at the moment of the shutdown, allowing the recalculation to resume from that same date)

  • If you start a long task that takes more than 1 hour, you should see an accumulation of « 30 minutes » tasks, I’ve added logs to be sure that the « Consumption » and « Cost » tasks calculate the same periods.

Build finished :
![image|690x51](upload://vzK6F

Thanks @Terdious je vais retester cette semaine, je viens seulement de me dépatouiller d’un problème de fuite mémoire qui dure depuis une semaine et qui me freezait le frontend :frowning:
And the worst part is that the culprit is a module NOUS B3Z that was sending 4 updates per second to my z2m and then putting Gladys in PLS (unresponsive), a nightmare to find :angry:
In the battle, my matter/matterbridge is down, so I’ll look at that part later.
Now it seems to be running correctly again (fingers crossed) so I’ll leave it like that for a bit so the Gladys Plus backups can restart quietly.
I’ll keep you posted as soon as I’ve tested your latest image with your requests.

So how did you reduce the number of requests from your NOUS plugs?

@_Will_71 I went to the module’s Settings tab and set a 10s delay for payload sends:
image
A reboot

First post of the year so best wishes for 2026 as it starts, and a nice piece of feedback for @Terdious :wink:

[quote="Terdious, post:146, topic:9854

And for the other points :

[quote=« mutmut, post:140, topic:9854 »]
when an item is selected, it appears in the analysis list but does not disappear from the selection listing (but you can’t

Hi @mutmut,

Thanks for your tests and feedback on this New Year’s Day ^^

It refers to the day currently in progress. So if you need to rerun, you must rerun that same day (this keeps the information consistent with the selection to make for the rerun). I can add "Ongoing Progress" rather than simply "Progress" for more clarity. To notice it, by displaying the day in question you can see that the calculation stopped during the day / or look at the previous day to see that all the hours of the day are properly filled ^^

:sweat_smile: Yes, I thought it could be useful during debugging to have enough info and I assumed Gladys handled a maximum log size. But indeed it might be too much …

:+1:

I just noticed that I had the same issue as @froch for Tempo schedules :
image
I should only have the first 6 bars as HC in my example (from 0:00 to 6:00) but I have HC values appearing at 6:00 even though I should be in HP.
I have the impression that if a value exceeds the end time and is recorded afterwards, it will be placed in the next time slot.
What’s strange is that the cost of this segment is correctly based on the blue HC rate (which stops at 6:00).

Following up on my post here about an erroneous value in one of my Tempo indexes (probably due to a data sending error from my teleinfo2mqtt system).

I just retrieved the values from 11/11/25 (with Yaak) to check what happened and there is an erroneous value that has been recorded (800 kWh too much) :frowning:


Is there a way to change/delete this value?

Is there no before/after value verification when calculating the energy monitoring so as to ignore/remove an erroneous value?
Of course, you would need to define what constitutes an erroneous value for Gladys…

And for the 30-minute consumption calculations, is it done on the sum of each value of each row, or as an average, or on each value every 30 minutes, etc.?

I also made a request to have a cleanup of unnecessary values for the indexes

Hi @mutmut

Good catch, indeed it looked very much like a bad value being recorded.

To modify or delete values, I personally use image DBeaver to open and view the database after stopping Gladys and backing up the DB.
Then I asked ChatGPT to write the query for me; I struggled with the format of the ‹ created_at › column so I won’t paste all its answers, but the overall query with before/after verification is:

SELECT device_feature_id, value, created_at
FROM t_device_feature_state
WHERE device_feature_id = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
  AND created_at >= TIMESTAMPTZ '2025-11-11 10:36:55.911+0000'
  AND created_at <  TIMESTAMPTZ '2025-11-11 10:36:55.912+0000';

UPDATE t_device_feature_state
SET value = 158021
WHERE device_feature_id = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
  AND created_at >= TIMESTAMPTZ '2025-11-11 10:36:55.911+0000'
  AND created_at <  TIMESTAMPTZ '2025-11-11 10:36:55.912+0000'
  AND value = 958021;

SELECT device_feature_id, value, created_at
FROM t_device_feature_state
WHERE device_feature_id = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
  AND created_at >= TIMESTAMPTZ '2025-11-11 10:36:55.911+0000'
  AND created_at <  TIMESTAMPTZ '2025-11-11 10:36:55.912+0000';

To execute the query in DBeaver (after opening the table — this configures the script directly on the correct DB / table) :


Then you paste the query into the window and replace the device_feature_id in each part of the query (I have already normally put the correct values and dates in the query based on your table — but check anyway ^^).
You can check what the query will act on before making changes by running only the 1st query. To do this place your cursor anywhere in the 1st query before the « ; » then run it with the 1st button :

And check at the bottom of the page if it’s the correct value/date

Finally, if it’s good, run everything with this button :

You will have the 2 select tabs Before/After at the bottom of the page :

That’s the problem… And while telling you my personal case, which is that, in this winter period, I consume up to 140kWh / day, I realized that @pierre-gilles had added a field in the energy tariffs that could be very interesting for detecting an inconsistency :
image

Indeed, it would just need to be specified that the value must be entered in kVA, and therefore as a number only. Then, we know the trip curves of a subscriber circuit breaker and in particular that it is impossible to operate at more than 18kWh + 10% i.e. 19.8kWh so over 30 minutes = 19.8 / 2 = 9.9kWh.

For this, we could integrate this mathematical formula to :

  • Not include higher values in calculations
  • Report and save values detected as inconsistent
  • In the future also integrate these inconsistencies for auxiliary devices (declare in a feature the max consumption values of a device like a dishwasher). I’m thinking of some Tasmota devices that have already reported values of ‹ 0 › on one update in 2024 and which will therefore have the same effect you just encountered.

It’s the sum of the differences between all successive values within the interval, it’s neither an average nor a punctual reading.

Finally, I’ve long agreed on having a way to verify/correct/include feature values directly in Gladys. But that represents a very big job, and I understand that it’s not obvious to develop. You can see it in HA, it’s heavy and I find in use not at all obvious/easy to use. So additionally you have to think of something very user-friendly while in the end very few will use it. That doesn’t necessarily make you want to work on it ^^

PS: On a personal project with friends, we also have this need, and we’ve been postponing the work on this part for years :grimacing: :sweat_smile:

I really like your idea of a max tied to the subscription that shouldn’t be exceeded, otherwise it would yield an incorrect value.

So I understand better why it’s so slow with my 6 indexes!
For the indexes that don’t move during the day, a single calculation would be necessary instead of +30000 per index per day :frowning:
Well it’s per 30-minute interval so my reasoning isn’t entirely accurate but if I go to 48 calculations that’s already huge!

What I have trouble understanding in this 30-minute calculation interval:

  • 10:36:53 → 158021
  • 10:36:55 → 958021, so a diff of +800000
  • 10:37:01 → 158021, so a diff of -800000

The total should be 0, right?
I remember a discussion about a potential hardware change but still linked to the same device. Could that be the famous Reset counter that prevents the sum from equaling 0?

That’s exactly it. Since there is a « negative value », it treats it either as a reset to 0, or the replacement of the counter by one that had a negative value.

You have to find the right balance between all uses and possible cases. In this situation, with an erroneous positive value returning to the correct value as you had, that shows us that not all scenarios have been considered.
So the solution to « best detect » an inconsistent value will indeed solve this case.

However, there will remain cases where a device returns erroneous values that are within the « safety » range we could put in place, but in that case it’s impossible to verify, so the only solution will remain in the hands of the users.

Indeed, it depends on whether he can modify this logic. But this is only really heavy for the initial calculations. For the subsequent tracking of 30-minute intervals, it doesn’t have a noticeable time impact.

No, the Matter integration doesn’t handle the new energy monitoring development — that’s a development task to be done!

I agree that an editor in Gladys allowing to view/modify sensor values would be great.

In any case, it’s much easier to develop than a pseudo-detection of inconsistent values, which will never cover all cases and doesn’t solve the issue of false values that would remain within the bounds.

Can you create a feature request?

In the meantime, @Terdious’s tutorial is great for making the modification directly in the DB! :slight_smile:

and there you go ^^ Visualiser/modifier/supprimer une valeur dans l'historique d'un élement