I’m following up on this topic — I’ve finished the verification for the PR that improves the energy tracking recalculation. I think you can review:
This will be followed by the next PR which adds clearer logging for the user, notably to see where the last launched tasks were at if a Gladys restart or an issue occurred during a recalculation (avoiding having to redo the entire recalculation …):
Could you describe in more detail what this PR does?
Since this touches the core, these are fairly sensitive changes.
I’d like to understand precisely the scope to be able to properly assess the impact.
During development I spent several months improving the reliability of the calculations, and it’s vital to me that the calculation isn’t affected
You’re right to ask for the details, I completely understand. And if possible, I’d prefer that you also test it by running it alongside your production environment to properly verify. That’s what I did to properly assess correct operation. I didn’t find any scenarios where it crashes. But for example, I don’t have any Zigbee energy. Even though there’s no reason it should be different, that would cover everything.
The overall diff is large (+3203 / -428), but the functional scope I’m targeting in this PR is mainly the following:
Targeted recalculation (backend) over a date range / feature selection
Added recalculation by date range for:
cost (calculate-cost-range)
consumption from index (calculate-consumption-from-index-range)
The “from beginning” endpoints now also accept a feature selection (feature_selectors).
Validation of YYYY-MM-DD dates on the controller side (with BadParameters if the format is invalid).
Partial recalculation without breaking the rest
When I recalculate a period, I only purge the states within the period (destroyStatesBetween) before recomputation.
When there is no end date, I keep the “from start → now” behavior (destroyStatesFrom).
For consumption from index, I restore ENERGY_INDEX_LAST_PROCESSED at the end of processing on partial/selected recalculations to avoid damaging the global cursor.
Execution as a background task
Switched to wrapperDetached for long recalculations (from beginning / range), to immediately return a job_id.
The frontend only considers the action successful if a job_id is returned.
Frontend (energy monitoring screen)
Added start date / end date fields (each field is optional).
Added multiple selection of features to recalculate.
If no feature is selected, explicit confirmation before a global recalculation.
Automatically call the range endpoints if a date is provided, otherwise the from-beginning endpoints.
What I did not touch in the calculation core
I did not change the contract calculation formulas (the engine contracts.calculateCost remains the same).
I did not change the basic logic of the index delta (the formula remains identical); I mainly constrained the scope (selectors, dates, targeted purge, job handling).
I have:
neither changed the business formulas of contract calculation
nor the principle of the index delta
nor the unit conversion we recently put in place.
I have « only » modified the input scope (selected features, date range) and the technical framing of the recalculations (targeted purge, jobs, API/front).
And to tell you the truth, at the start of the week I almost closed the PRs for this topic and wrote to tell you that I was leaving it in your hands.
I was afraid it would take you longer to review it than to do it yourself. That on top of that visually it might not suit you. And that as a result you’d probably be better off doing it the way you think.
But I’m torn between that and the fact that there are still a few of us waiting. And the fact that I can see you’re busy with many other things.
So I decided to take the approach of proposing it and leaving it up to you. And you’ll give me your feedback / impressions.
