Hello! I tried creating scenes, but it didnât work out:
The log is huge, but the last line says⊠> data: { status: 500, error_code: âč SERVER_ERROR âș } Edit: It crashed⊠Iâll leave it alone. Itâs Saturday, maybe itâs the weekend for it too!
Thanks for the feedback, itâs funny your prompt made the AI completely hallucinate, all its attempts were wrong (device_get_state instead of creating a scene)
Try sending a message again but maybe be more explicit?
Well, Iâve rephrased, discussed, argued but to no avail.
Hereâs a small excerpt
There were several issues. First, I have a bathroom and a main switch that share the same name. Then, devices with multiple features were problematic. Finally, it didnât understand how to âč wait âș.
In the end, thereâs always something that doesnât work, but I havenât found out what yetâŠ
Edit:
Then I have the list of all the devices in the house
ending with
Thanks for this detailed feedback!
Iâll check if thereâs a modeling issue in the scene schema that could explain this bug.
Useful tip: the AI is limited to 5 consecutive autonomous iterations to avoid infinite loops. So when tool calls stop and donât resume, thereâs no need to wait: if you no longer see « Gladys is writing », thereâs no background processing going on.
As soon as a response from Gladys appears, it means the execution is complete. In your example, we often see 4 tool calls followed by a final response.
Generally speaking, is your instance large? How many devices? How many scenes?
I wonder if the model isnât hallucinating a bit in your case, because in any case, it should create a scene and not device_get_stateâŠ
So thereâs no point in waiting. Iâll make a note 
There are about fifty sensor and switch values and about fifty scenes (most are very short).
I tried again this morning. I had to restart Gladys because the chat wasnât responding. Then, indeed, 4 get device states (the last one isnât useful) and then nothing. The chat no longer responds to my messages. Gladys isnât actually crashed.
This time, nothing appeared in the logs.
Nothing is urgent, have a good Sunday!
Iâm investigating tomorrow!
@GBoulvin I managed to reproduce the issue, and Iâve fixed the bug. It will be included in the next version of Gladys. The scene schema sent to the AI wasnât clear enough 
In addition, I improved error handling. If the AI model sends an empty response, an error message will be displayed in the chat! This will prevent the « non-responses » you experienced
I asked the AI by voice to create a scene with sunset as the trigger, with a 20-minute delay for turning on an LED strip. In the end, the scene doesnât include the action, even though it was mentioned in the response message. Is this normal?
@StephaneB I fixed that, it will be in the next version of Gladys:
Easier to see what the AI is doing via Telegram
Good evening everyone,
The forum is very active, and thatâs great. I canât keep up with everything anymore, so I have a question about AI. As it has made a lot of progress, I wanted to know if Gladys could retrieve values read by a camera interpreted by the AI to send them to Node Red? Basically, Iâm trying to retrieve the values of my pH and Redox by asking the AI to analyze my camera, which is pointed at my devices. I ask the AI to read the values (which works quite well) and sends them back to me on Telegram. I would have liked to retrieve them in Node Red to be able to threshold them. Is this possible? 
In my opinion, you shouldnât send them directly to Telegram. I will go through one or more MQTT devices and then send them via Telegram.
Hi @Psoy
This isnât possible right now, but it should be fairly easy to add!
I think we could give the AI a new tool to set a sensor value, which you could then read in the scene and send to Node-RED. What do you think?
If you like the idea, could you create a feature request?
good evening @pierre-gilles
thanks for your reply, but what do you mean by: giving the AI a new tool to define a sensor value? ; in my case, it would be like reading a license plate, interpreting it and comparing it with a list of authorized plates to open a gate for example with a scene. Basically, itâs about retrieving a value read by a camera and making it a variable that we can test. I mentioned Node-RED, but with a scene, that would already be great
Good evening @mutmut
I didnât express myself well, I meant, when I use the « ask the AI » function in the scenes, that the values were sent to Telegram, in the message. I agree with you, we should create an Mqtt device to retrieve these values. I described in the following message what I wanted to do more clearly, I hope. Thanks for your answer
Gladysâs AI agent has a set of tools that allow it to interact with your Gladys instance.
This is what you see in the chat when you ask it a question, for example:
Here, the action device_get_state is a tool that the AI used to retrieve the state of a device before answering you.
We could imagine adding a new tool, for example sensor_set_state, which would allow the AI to write a value to a virtual sensor.
In this case, your scene could work like this:
- Ask the AI to read the pH value displayed by the camera, then save it in a virtual sensor
pH-camera-readingthat you would have created beforehand via the Virtual Device MQTT integration. - Retrieve the value of this sensor in Gladys.
- Perform the processing of your choice: comparison with a threshold, sending a notification, triggering a scenario, etc.
This seems to be an interesting use case. If you like the idea, I invite you to open a feature request to add this new tool to the AI agent 
Could this use case also be used for a thermostat for example? This would allow you to adjust it directly 
Thanks @pierre-gilles.
Explained like that, I understand and thatâs exactly it,
Here, the action `device_get_state` is a tool that the AI.
I always use Telegram and I hadnât seen this line in the discussion
Iâm making a feature request,
Iâll take inspiration from your explanation to make the feature request, it will be clearer
Thanks again 
.
Hello @guim31
I imagine that after reading your thermostat with a camera, the virtual sensor retrieves this value. After that, I think we can manipulate it, increment it, threshold it, or pass it as an argument in « control a device »