Alternatives for scenes interrupted by updates

EDIT: discussions moved from Gladys Assistant 4.85.0 : widget Photo, navigation dans les graphiques, chauffe-eau & HomeKit - #21 par Will_71 to dedicate a specific topic.


Bravo to everyone for all these new features and improvements :heart_eyes:

I have a remark that almost bothers me with every update: the scenes that have an Wait in progress.
To make it simple, I start the pool pump for a duration that I define and therefore I have a Wait action for X hours. Unfortunately, when an update occurs, my Wait simply skips and if I’m not careful, the pool can run for 8 hours instead of 3 hours :frowning:
Apart from setting up MQTT variables to track an ongoing scene (which I should do), creating timestamps that I also store, and a script on (re)start of Gladys (by the way, does a boot trigger exist?), wouldn’t there be a (new?) something to resume these scenes where they were stopped?

Thanks for the feedback, and you’re pointing out a real limitation of the current implementation :slight_smile:

Technically, the « Wait » block is just a timer in memory in the Gladys process. Nothing is stored in the database: neither the fact that a scene is running, nor where it is in its list of actions. So, at the slightest restart (update, reboot, power outage), everything after the Wait is lost. It’s not a bug on your side, it’s just how it is today.

And yes, the « Gladys starts » trigger does exist, you’ll find it in the list of triggers.

But I would rather advise you to organize yourself so that you don’t need it.

The smartest thing in your case is to have the timer handled not by Gladys at all, but by the outlet itself. Most hardware can do « turn on and turn off by itself after X seconds ». It depends on what you have:

  • Shelly, with the HTTP Request action: http://<ip>/relay/0?turn=on&timer=10800 (Gen1) or http://<ip>/rpc/Switch.Set?id=0&on=true&toggle_after=10800 (Gen2+)
  • Zigbee2MQTT, with the Zigbee2MQTT action: on zigbee2mqtt/<your_pump>/set, you send {"state":"ON","on_time":10800,"off_wait_time":0}. It works on many Tuya/Moes/Aqara outlets and modules.
  • Tasmota: PulseTime 10900; Power ON (beyond 111, PulseTime equals value − 100 in seconds)

Your scene becomes a single action, no more Wait. And the huge advantage for a pool pump: even if your Gladys is cut off or the mini-PC doesn’t restart, the pump still stops.

If your hardware can’t do that, the second approach is a countdown rather than a wait.

You create a number feature via MQTT, like pool_pump_minutes_remaining. Your startup scene turns on the pump and sets the value to 180. Next to it, you make a monitoring scene with a scheduled trigger in interval mode every 5 minutes, which retrieves the value, continues only if it is > 0, rewrites it with the formula {{0.last_value}} - 5, and if it drops to 0 it turns off the pump. The formula mode works just as well on « Set a value » as in conditions, so everything is done with the mouse.

The interest is that it repairs itself: no need for a boot script or the startup trigger, the next cron run catches up on the situation. At worst, you lose the 5 minutes of the current tick.

And if you can settle for that, the simplest solution is two scheduled scenes, start at 10:00 a.m. and stop at 1:00 p.m. For a pool pump, this is often sufficient and it’s foolproof. If you adjust the duration to the water temperature, you can even have 3 stop scenes (12:00 p.m., 1:00 p.m., 2:00 p.m.) each with a condition on the temperature, it’s a bit brute force but it always works.

The principle to remember: the Wait is for short delays within a sequence, like 2 seconds between two commands or 30 seconds before checking a state. As soon as you exceed a few minutes, and especially when a shutdown action is involved, you need to either offload the timer to the hardware or use a stored state that is read regularly.

That said, your original request remains legitimate: persisting long Waits to reschedule them on startup is doable. If you want to open a feature request about it, don’t hesitate.

Excellent method I hadn’t thought of at all.
I have a NodOn SIN-4-1-21 and it seems possible:

On with timed off

When setting the state to ON, it might be possible to specify an automatic shutoff after a certain amount of time. To do this add an additional property on_time to the payload which is the time in seconds the state should remain on.

Additionally an off_wait_time property can be added to the payload to specify the cooldown time in seconds when the switch will not answer to other on with timed off commands.

Support depends on the switch firmware. Some devices might require both on_time and off_wait_time to work

Examples : {"state" : "ON", "on_time": 300}, {"state" : "ON", "on_time": 300, "off_wait_time": 120}.
And as I send myself a message at the end of the Wait, I will have to create a monitoring scene that checks every minute if the module is on or off.

Thanks for the tip!

I tried with the variable calculation {"state":"ON","on_time": 1.1. Pool (filtration_cycle_duration) *3600 ,"off_wait_time":0} and apparently it didn’t work :frowning:

Do I need to go through a temporary MQTT variable?
And the thing is, I have no way to see the command sent (nothing in Gladys logs, nothing in Z2M).

EDIT: eventually {"state":"ON","on_time":10800,"off_wait_time":0} doesn’t work either :frowning:

After some tests, you should not take the action Send a zigbee2mqtt message but Send an MQTT message otherwise it doesn’t work.
@pierre-gilles is there a specific difference between the 2 actions?

I tested {"state":"ON","on_time":60,"off_wait_time":10} in MQTT (not zigbee2mqtt) and it works, 1 minute later the pump stops properly.

I tested {"state":"ON","on_time":10800,"off_wait_time":10} and there is an issue, either on Gladys’ side or z2m, because everything is multiplied by 10 and therefore I have an out of range:

z2m: Publish 'set' 'state' to 'relais_03' failed: 'RangeError [ERR_OUT_OF_RANGE]: ZCL command 0x84b4dbfffe08ce6e/1 genOnOff.onWithTimedOff({"ctrlbits":0,"ontime":108000,"offwaittime":100}, {"timeout":10000,"disableResponse":false,"disableRecovery":false,"disableDefaultResponse":false,"direction":0,"reservedBits":0,"writeUndiv":false}) failed (The value of "value" is out of range. It must be >= 0 and <= 65535. Received 108000)'

Any ideas?

I continue my tests and the variables don’t work:


and in the z2m logs:

z2m: Invalid message 'undefined', skipping...

not very explicit…

Thank you for all these tests, you’ve identified several issues, and at least one of them is a bug on our side. I’ll address them in order.

1. The difference between the two actions

In the code, Send an MQTT message and Send a Zigbee2mqtt message do exactly the same thing: your text goes through the variable engine and is then published as is. The only difference is the MQTT client used: the MQTT action publishes to the broker configured in the MQTT service, the Zigbee2mqtt action publishes to the dedicated connection of the Zigbee2mqtt service (by default the Mosquitto dedicated to Z2M, mqtt://localhost:1884).

If your Z2M runs outside of Gladys with your own broker, the Zigbee2mqtt action therefore potentially goes nowhere: no one is listening, and since the publication log is at debug level, you indeed see nothing anywhere. It’s not you who missed something. Also note: this action adds no prefix zigbee2mqtt/, you need to write the full topic zigbee2mqtt/relais_03/set — the placeholder gladys/my-topic is misleading, we’ll fix it.

In short: in your config, stick with Send an MQTT message, it’s the right choice.

2. The ×10 and the RangeError

This is neither Gladys nor a Z2M bug: it’s the Zigbee protocol. The ZCL command genOnOff.onWithTimedOff expresses ontime and offwaittime in tenths of a second, on a 16-bit integer. Zigbee2MQTT therefore does on_time × 10, and the ceiling is 65535.

In other words, the absolute maximum is 6553 seconds, or 1 hour 49 minutes. Your 3 hours will never pass in a single command, no matter the path used. That’s why your 10800 also failed with the Zigbee2mqtt action, in addition to the broker issue.

3. Variables in the payload

The message field accepts variables, but with two important limitations:

  • the syntax is {{ }} (type {{ in the field, autocomplete appears), and the variable must come from a Retrieve the last state action placed before, in a previous group;
  • no calculation is possible in this field. Unlike Set a value or conditions, there is no formula engine here: {{0.0.last_value}}*3600 will be sent literally as 3*3600, not as 10800.

And when the variable is not resolved, it is replaced by an empty string, which produces a broken JSON like {"state":"ON","on_time":,"off_wait_time":0} — hence the rejection on the Z2M side. We should clearly log the payload actually published, it’s a real debug gap. Meanwhile, to see what really goes out:

mosquitto_sub -h <your_broker> -t 'zigbee2mqtt/relais_03/set' -v

To do your multiplication, you need to take it out of the payload: a Set a value action in formula mode on an MQTT number feature (pompe_on_time), with the formula {{...}}*3600, then a Retrieve the last state on this feature, and finally {{x.y.last_value}} in the JSON.

4. A bug on our side regarding formulas

While checking this point, I discovered that our formula engine is instantiated without the subtract function: subtraction does not work. The formula {{0.0.last_value}} - 5 that I gave you earlier raises an exception, which is silently swallowed (the action fails, the scene continues, nothing happens). My apologies, it was wrong.

In the meantime, the workaround is to write {{0.0.last_value}} + -5, which works. +, *, /, round() and mod are OK.

Ticket created: Scene formula engine: subtraction is not supported (`Function subtract missing in provided namespace "math"`) · Issue #2823 · GladysAssistant/Gladys · GitHub

5. What I finally advise you: the countdown + watchdog

Given the 1 hour 49 minutes ceiling, the right approach combines both ideas, and it’s even better than what I initially proposed, because it remains unbreakable in case of Gladys outage.

Create an MQTT number feature pompe_minutes_restantes.

Scene « Start the pump »:

  • Set a value: pompe_minutes_restantes = 180
  • Send an MQTT message to zigbee2mqtt/relais_03/set: {"state":"ON","on_time":900,"off_wait_time":0}

Scene « Pump monitoring », scheduled trigger in interval every 10 minutes:

  • Retrieve the last state of pompe_minutes_restantes
  • If / Else on > 0:
    • then: Set a value = {{0.0.last_value}} + -10, then republish {"state":"ON","on_time":900,"off_wait_time":0}
    • else: publish {"state":"OFF"}

The principle: the relay never receives more than a 15-minute authorization, which the monitoring scene renews as long as the countdown is positive. As a result, if Gladys is down for an update, a reboot, or a crash, the pump stops on its own after a maximum of 15 minutes, and upon restart, the countdown resumes where it left off since it is stored in the database. No more Wait, and nothing to fix at boot.

Keep off_wait_time at 0: it’s a guard time during which the module refuses new on with timed off commands, it would prevent rearming.

6. Your supervision scene every minute

Not needed. The NodOn reports its state change to Z2M when it turns off on its own, so a scene with the trigger A device changes state on the relay feature, condition « is off », is enough to send you your end notification.

I’m creating the tickets for the missing subtraction, the log of the published payload, and the Z2M topic placeholder.

Thanks @pierre-gilles for the explanations, I understand the difference between MQTT and Z2M actions.
In my case, I have the Z2M integration external with an external MQTT, and the MQTT integration of Gladys points to the external MQTT, so the 2 actions will go and pick from the same final basket.
This means that if I use one or the other of the commands, I should get the same result, but that’s not the case because it works with the MQTT action and not Z2M.

I see that Claude has answered because in some statements I see some wolves :face_with_raised_eyebrow:

And what I don’t understand at all is that I tested with 60 (turn off after 1 minute), 1000 (turn off after about 16 minutes) and 3600 (turn off after 1 hour), and none of these numbers were multiplied by 10 since the turn-offs were correct..

Yes but.
I do have an intermediate step to avoid the calculation:




And the message sent to me indicates the recalculated time in seconds, but it doesn’t work in the payload.
And I test with 1 hour to be sure not to have an out of range.

I will look at point 5 in more detail, it seems like a good idea, and I will be careful with the subtraction for now.

I’m sorry, I’m just flat out with Claude, but with all the work on external integrations, I can’t offer you anything better :smiley:

You’re right on both points, and on one of them, I was the one who made a mistake. I’ll take it back.

The ×10: your tests confirm the explanation, they don’t contradict it.

My wording was bad. The ×10 is not something that « adds » to your duration, it’s a unit conversion: the ZCL field is expressed in tenths of a second, so Zigbee2MQTT converts your seconds into tenths before sending, and the module reconverts upon arrival. Result, your duration is well respected — it’s exactly what you measured:

on_time value sent turn off
60 600 1 min :check_mark:
1000 10000 16 min 40 s :check_mark:
3600 36000 1 h :check_mark:
10800 108000 > 65535 → RangeError

The code is here, in the Z2M converters:

{ctrlbits: 0, ontime: Math.round(onTime * 10), offwaittime: Math.round(offWaitTime * 10)}

The only time you see it is when the result exceeds 65535, the limit of a 16-bit integer — and that’s exactly the error message you got. The ceiling is therefore 6553 seconds, or 1 hour 49 minutes. Your 3600 pass, your 10800 will never pass. (By the way, Z2M bounds this value for other parameters of the same type but not for on_time, hence the brutal error rather than a silent ceiling; it would deserve a ticket with them.)

The log Invalid message 'undefined' doesn’t mean what we think.

I looked at the Z2M code:

const message = this.parseMessage(parsedTopic, data);

if (!message) {

logger.error(`Invalid message '${message}', skipping...`);

It displays the result of the parsing, not the received message. As the parsing failed, this result is always undefined. In other words, this log will say undefined no matter the payload: it’s not Gladys who sent you the word undefined. The only useful information is « the received JSON was not valid ». That’s why I told you to go through mosquitto_sub: it’s today the only way to see the real payload.

The editor, on the other hand, is out of the question.

I checked the track of a text damaged by the variable field (invisible characters, re-encoded quotes): I replayed the component in a browser with your configuration, the text comes out identical to the character, including curly braces and quotes. That’s not where it breaks.

On the other hand, I found a real trap, and it could be yours.

If the variable is the last element of your JSON, you end up with three curly braces stuck together:

{"state":"ON","on_time":{{1.1. Pool ...}}}

This }}} is interpreted as the closing of another variable syntax, the template engine crashes, and the error is silently swallowed: nothing is published at all. Two immediate workarounds, both tested:

  • put a space before the final curly brace: ..."on_time":{{variable}} }
  • or simply not end with the variable: {"on_time":{{variable}},"state":"ON","off_wait_time":0}

What I need to settle your case

The captures don’t allow me to see the exact characters. Can you copy-paste, in text:

  1. the exact content of the message field of your MQTT action;
  2. the exact topic;
  3. and if possible the output of mosquitto_sub -h <your_broker> -t 'zigbee2mqtt/relais_03/set' -v while you launch the scene.

With that, we’ll know in thirty seconds if it’s the }}}, an empty variable, or something else.

On MQTT vs Z2M with the same broker: you’re right, my explanation doesn’t hold.

If both point to the same external broker, both actions should indeed produce the same result. Two leads remain:

  • the topic. The Zigbee2mqtt action adds no prefix, contrary to what its name suggests. Did you write zigbee2mqtt/relais_03/set in both actions? If in the Z2M action you had put relais_03/set, that would explain everything.
  • the connection. The Zigbee2mqtt service opens its own MQTT session, with the credentials entered in its integration — distinct from those of the MQTT service. If these credentials are refused by your broker, the MQTT library queues the messages without ever raising a visible error. Is your Zigbee2mqtt integration displayed as connected in Gladys?

No worries @pierre-gilles, I know you have a lot on your plate right now and you need an assistant :wink: ! … Gladys? No, Claude :sweat_smile:

It’s good, I finally understood how it works and so before reconverting in Z2M, there is the fatal barrier of 65535 not to exceed. All good!

  1. copy/paste as is:
{"state":"ON","on_time":

2.1.A.2.1. Pool (filtration_duree_cycle_secondes)

 ,"off_wait_time":0}

but it’s not possible to give the exact content because we no longer have the curly braces once but I have a space after the variable:
image

  1. zigbee2mqtt/relais_03/set
  2. mosquitto_sub -h <your_broker> -t 'zigbee2mqtt/relais_03/set' -v
    I have no idea where to type this command because in the Gladys docker it tells me unknown command :frowning:

no shortcut, I have the full topic with zigbee2mqtt/relais_03/set.
And I had tested both just in case :wink:

Everything is absolutely the same in terms of login/password configurations in MQTT and Z2M, and everything is operational otherwise I would have issues controlling all the other devices:


Ok, is everything good on your end then? :slight_smile: You can mark the topic as « Solved » if so :wink:

not yet resolved :frowning:

I don’t know if you have a Claude account, but it might be faster if you talked to them directly :joy:

Here’s Claude’s response:

Found. And it’s not you, it’s us — your copy-paste contained the answer, you just had to look at the characters one by one.

Your payload contains a space that isn’t one.

When you insert a variable from the list, the editor automatically adds a space after it — that’s the one you noticed. Except the browser doesn’t put a regular space: it puts a non-breaking space, the character U+00A0. Visually it’s identical, even in a copy-paste, but for a JSON parser it’s an invalid character in that position.

So your payload goes out like this, with a non-breaking space where I’ve put the symbol:

{"state":"ON","on_time":3600␣,"off_wait_time":0}

JSON.parse refuses, parseMessage returns undefined, and Z2M logs its famous Invalid message 'undefined'. The loop is complete.

I replicated your exact input in a browser, with your variable and payload: I reproduce the character identically. And that finally explains everything else:

  • your notification with the same variable displays correctly — a non-breaking space in a sentence doesn’t show and doesn’t bother anyone;
  • your test from post #5 with on_time:60 hardcoded worked — no variable, so no added space, so no non-breaking space;
  • and nothing could appear in the logs, neither for us nor for Z2M.

The workaround, right away

In your message field: place your cursor right after the variable and press Backspace once to delete the space. You should get the variable directly next to the comma. I checked, the payload becomes valid again and the command goes through.

Also keep in mind the second trap: don’t end your JSON with the variable, otherwise the three curly braces stuck together }}} crash the template engine. Always put something after, for example ,"off_wait_time":0}. In your case it’s already good.

We fix it

This is a clear bug on our side: the editor should never let that character pass. The fix is trivial, we normalize the non-breaking space when the text is saved. It will be in a next version, and in the meantime your Backspace is enough.

For mosquitto_sub

Normal that it doesn’t work in the Gladys container, the tool isn’t installed there. Three options depending on what suits you:

  • from your broker container: docker exec -it <mosquitto_container_name> mosquitto_sub -t 'zigbee2mqtt/relais_03/set' -v
  • on the host machine, after a sudo apt install mosquitto-clients
  • or, simpler if you prefer the graphical interface, MQTT Explorer, which shows you all the broker traffic live

That said, you don’t need it for this issue anymore.

There’s still one point unresolved

Your observation from post #5 — the MQTT action works, the Zigbee2mqtt action doesn’t, with the same complete topic and the same broker — still isn’t explained, and the non-breaking space doesn’t change anything since your test was with fixed values. Now that we know this field can contain invisible characters, can you try a clean test: two actions, the payload {"state":"ON","on_time":60,"off_wait_time":10} retyped entirely by hand in each, same topic zigbee2mqtt/relais_03/set? If the Zigbee2mqtt action still fails, we have a second bug to dig into and I’ll open a ticket on it.

Fixed in Gladys Assistant 4.86:

I finally did some new tests and it works a lot, even in z2m, great!

However, there is still a space that gets added at the end of the variable you insert:


And this space messes up the json sent to z2m which receives a bad message → z2m: Invalid message 'undefined', skipping...

So you must absolutely remove this space between the variable and the comma and then everything works correctly.
I will create an improvement request.

EDIT: [Scènes] supprimer l'espace inséré automatiquement après l'appel d'une variable