Hi @StephaneB, sorry for the delay.
On the subject, I won’t go for a list of pre-written phrases drawn at random. The direction of Gladys is AI: a generated and contextual response, not a fixed dictionary. So the issue to address here isn’t « how to bypass the AI, » it’s « why the AI takes 15 seconds on your end. »
And 15-20 seconds is not normal. To be transparent about the architecture: the « Ask the AI » action doesn’t make a single call. First, there’s a quick intention classification call, then the main call, and then one or more additional round trips if the model decides to call a tool (e.g., to read a temperature). But even with 3 round trips, we should be around 1.5-2 seconds, not 15. So something specific to your installation is happening.
I don’t have time to investigate myself, but the good news is that you have everything you need to do it without waiting for me — without a command line, and Gladys already records everything necessary. Here are the keys.
1. See exactly what the AI did, tool call by tool call
This is the core of the diagnosis. Trigger your scene, wait for the voice response, then go to Integrations → Artificial Intelligence, section « AI Debug » → « Download JSON context for debugging. »
Open the obtained file (a text editor is sufficient) and look at the _debug → conversationHistory section. It contains two things:
toolCalls: the exact list of all the tools the AI called, with for each its name (tool_name), its result (tool_status: success or error), the arguments passed to it, and the exact time (created_at).messages: the complete conversation in chronological order — your message, each tool call, the final response — each entry timestamped.
By subtracting the created_at, you see precisely how many tools were called during your scene execution, which ones, and which one took the time. Two signals to watch for:
- multiple tool calls for a simple confirmation phrase → each call is a complete round trip to the model, that’s probably where your seconds are going;
- a
tool_status: "error" → a tool that fails makes the model restart for an additional round, which can double or triple the total time. This is the kind of thing you don’t suspect and that jumps out in this file.
The calls triggered by a scene are well recorded here: the history is that of the user chosen in the « Ask the AI » action. So make sure to trigger the scene before downloading the file.
Incidentally, the tools section of the same file lists all the tools sent to the model with each call. If it is very large (many devices), it also weighs down each request.
2. Supplement with logs if needed
If the file above is not enough to decide, Settings → System → Download logs, then search for [AI_CHAT] in the file. You get the server-side trace, timestamped line by line:
[AI_CHAT] New request userId=... message=...
[AI_CHAT] Forcing tool_choice=required for categories=device_query
[AI_CHAT] Assistant turn iteration=1 tool_calls=1 tool_choice=required tools=[...]
[AI_CHAT] Running tool=...
[AI_CHAT] Tool finished tool=... status=success resultLength=...
[AI_CHAT] Assistant turn iteration=2 tool_calls=0 ...
[AI_CHAT] Completed answerLength=...
The reading grid:
| Where time is spent |
What it means |
Between New request and the 1st Assistant turn |
Local preparation (Gladys rebuilds the list of tools from your database) + classification call |
Many lines Assistant turn iteration= |
The number of round trips to the model explodes |
Between Running tool= and Tool finished tool= |
The execution of the tool on your instance that is slow |
A single Assistant turn, but 10 s to reach it |
The model or your network connection |
3. An immediate comparative test
In the chat with Gladys, successively ask your complete prompt, then a short version without any reference to a sensor: « Give me a short phrase to confirm that you are handling the request. » Time both with a stopwatch.
If the short one responds in 2 seconds and the long one in 15 seconds, it’s confirmed: it’s your prompt that triggers tool calls. Mentioning the water temperature makes the model understand that it needs to go read a sensor before responding. In this case, you have an immediate gain by rewriting your prompt so that it doesn’t need any data — and the file from step 1 will show it to you in black and white.
Let an AI help you analyze all this
This is exactly the kind of task where Claude (or your favorite assistant) is very effective, and it saves you from waiting for me. Give them the content of _debug.conversationHistory with an instruction like:
« Here is the timestamped history of an AI call in Gladys Assistant, in JSON format. List the tools called in order, calculate the duration between each step, and tell me where the majority of the total time is spent. Flag any tool in error. »
Gladys is open source, so you can also give them the file that handles all this flow so they can reason on the actual code: server/lib/gateway/gateway.forwardMessageToAiChat.js in the project’s GitHub repository.
Small precaution: this file contains your messages, your room and device names. Remove what you don’t want to share before pasting it anywhere, here or elsewhere.
It’s actually a topic that is already planned and even partly developed.