Gladys becomes unreachable after a few hours of operation

StephaneB · June 22, 2024, 11:12pm

Hello,

I would need your expert advice to find the cause of a failure on my Gladys system.

To describe it:

for about 2–3 weeks, Gladys becomes unreachable after a certain time of normal operation, to the point that I have to force a container restart (docker container restart gladys)
this hang occurs after a variable duration, roughly between 10h and 30h. The crashes don’t happen at a fixed time. Sometimes at night, sometimes during the day…
the symptoms of the hang are quite variable: sometimes the default dashboard loads, but if I try to switch dashboards a loading wheel spins endlessly; sometimes the dashboard doesn’t display at all
connection via Gladys Plus tells me my instance is not running, which makes sense I think since the Gladys server is ‹ stalled ›
the Raspberry Pi running Gladys is not saturated, but I detect an increase in CPU usage: usually between 5% and 30%, rising to 40% when these hangs occur
when Gladys is stuck, I can connect via ssh without issue and run commands. That’s how I restart the container to restore things.

Gladys runs on an RPi 4 (Raspberry Pi 4), with an SSD that’s about 45% used.

I have quite a lot of hardware connected to Gladys (about 50 Zigbee devices, for example) and a good number of scenes (approx. 250). But I don’t recall making any particular change before these stalls started.

Which commands do you think I should run at the next hang to collect relevant logs?

Thanks in advance for your help!

cicoub13 · June 23, 2024, 12:58pm

The docker stats command lets you see which container is consuming a lot of resources and therefore confirm (or rule out) that Gladys

StephaneB · June 23, 2024, 7:52pm

Thanks @cicoub13 for the tips.
And without waiting for another blockage, I launched the two searches in the last 10 minutes of logs, and they already bring up a lot of things:

« … grep SQLITE_BUSY » returns:

pi@domotique:~ $ docker logs gladys --since 10m | grep SQLITE_BUSY
2024-06-23T21:32:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:32:12+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:32:21+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:32:31+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:32:40+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:32:50+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:32:59+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:33:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:33:09+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:33:18+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:33:28+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:33:37+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:33:47+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:33:56+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:34:02+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:34:06+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:34:15+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:34:25+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:34:34+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:34:44+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:34:53+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:35:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:35:03+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:35:12+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:35:22+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:35:31+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:35:41+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:35:50+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:35:50+0200<warn> scene.executeActions.js:37 (executeAction) SyntaxError: Value expected (char 1)
    at createSyntaxError (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1623:17)
    at parseEnd (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1590:13)
    at parseParentheses (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1577:12)
    at parseNumber (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1550:12)
    at parseObject (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1534:12)
    at parseMatrix (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1466:12)
    at parseSingleQuotesString (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1382:12)
    at parseDoubleQuotesString (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1338:12)
    at parseSymbol (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1237:12)
    at parseCustomNodes (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1210:12)
    at parseLeftHandOperators (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1134:12)
    at parsePow (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1116:12)
    at parseUnary (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1105:12)
    at parsePercentage (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1065:12)
    at parseRule2 (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1016:16)
    at parseImplicitMultiplication (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:988:12)
    at parseMultiplyDivide (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:958:12)
    at parseAddSubtract (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:931:12)
    at parseRange (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:893:14)
    at parseConversion (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:858:12)
    at parseShift (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:835:12)
    at parseRelational (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:797:19)
    at parseBitwiseAnd (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:783:16)
    at parseBitwiseXor (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:768:16)
    at parseBitwiseOr (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:753:16)
    at parseLogicalAnd (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:738:16)
    at parseLogicalXor (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:723:16)
    at parseLogicalOr (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:708:16)
    at parseConditional (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:679:16)
    at parseAssignment (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:632:16)
    at parseBlock (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:580:14)
    at parseStart (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:551:16)
    at string (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:75:14)
    at parse (/src/server/node_modules/typed-function/lib/esm/typed-function.mjs:1790:20)
    at string (/src/server/node_modules/mathjs/lib/cjs/expression/function/evaluate.js:50:14)
    at evaluate (/src/server/node_modules/typed-function/lib/esm/typed-function.mjs:1790:20)
    at /src/server/lib/scene/scene.actions.js:264:17
    at Array.forEach (<anonymous>)
    at Object.condition.only-continue-if (/src/server/lib/scene/scene.actions.js:261:23)
    at executeAction (/src/server/lib/scene/scene.executeActions.js:32:35)
    at /src/server/lib/scene/scene.executeActions.js:66:13
    at tryCatcher (/src/server/node_modules/bluebird/js/release/util.js:16:23)
    at MappingPromiseArray._promiseFulfilled (/src/server/node_modules/bluebird/js/release/map.js:68:38)
    at MappingPromiseArray.PromiseArray._iterate (/src/server/node_modules/bluebird/js/release/promise_array.js:115:31)
    at MappingPromiseArray.init (/src/server/node_modules/bluebird/js/release/promise_array.js:79:10)
    at MappingPromiseArray._asyncInit (/src/server/node_modules/bluebird/js/release/map.js:37:10)
    at _drainQueueStep (/src/server/node_modules/bluebird/js/release/async.js:97:12)
    at _drainQueue (/src/server/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/src/server/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues (/src/server/node_modules/bluebird/js/release/async.js:15:14)
    at processImmediate (node:internal/timers:476:21) {
  char: 1
}
2024-06-23T21:35:51+0200 <warn> scene.executeActions.js:37 (executeAction) Error: Undefined symbol Clouds
    at Function.onUndefinedSymbol (/src/server/node_modules/mathjs/lib/cjs/expression/node/SymbolNode.js:222:15)
    at /src/server/node_modules/mathjs/lib/cjs/expression/node/SymbolNode.js:96:99
    at Object.evaluate (/src/server/node_modules/mathjs/lib/cjs/expression/node/Node.js:77:18)
    at string (/src/server/node_modules/mathjs/lib/cjs/expression/function/evaluate.js:50:36)
    at evaluate (/src/server/node_modules/typed-function/lib/esm/typed-function.mjs:1790:20)
    at Object.device.set-value (/src/server/lib/scene/scene.actions.js:57:15)
    at executeAction (/src/server/lib/scene/scene.executeActions.js:32:35)
    at /src/server/lib/scene/scene.executeActions.js:66:13
    at tryCatcher (/src/server/node_modules/bluebird/js/release/util.js:16:23)
    at MappingPromiseArray._promiseFulfilled (/src/server/node_modules/bluebird/js/release/map.js:68:38)
    at MappingPromiseArray.PromiseArray._iterate (/src/server/node_modules/bluebird/js/release/promise_array.js:115:31)
    at MappingPromiseArray.init (/src/server/node_modules/bluebird/js/release/promise_array.js:79:10)
    at MappingPromiseArray._asyncInit (/src/server/node_modules/bluebird/js/release/map.js:37:10)
    at _drainQueueStep (/src/server/node_modules/bluebird/js/release/async.js:97:12)
    at _drainQueue (/src/server/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/src/server/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues (/src/server/node_modules/bluebird/js/release/async.js:15:14)
    at processImmediate (node:internal/timers:476:21)
2024-06-23T21:36:00+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:36:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:36:09+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:36:19+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:36:28+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:36:38+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:36:47+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:36:57+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:37:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:37:06+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:37:16+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:37:25+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:37:35+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:37:44+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:37:54+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:38:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:38:03+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:38:12+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:38:22+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:38:32+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:38:41+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:38:51+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:39:00+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:39:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:39:10+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:39:19+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:39:29+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:39:38+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:39:48+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:39:57+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:40:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:40:07+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:40:16+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:40:26+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:40:35+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:40:45+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:40:50+0200<warn> scene.executeActions.js:37 (executeAction) SyntaxError: Value expected (char 1)
    at createSyntaxError (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1623:17)
    at parseEnd (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1590:13)
    at parseParentheses (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1577:12)
    at parseNumber (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1550:12)
    at parseObject (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1534:12)
    at parseMatrix (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1466:12)
    at parseSingleQuotesString (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1382:12)
    at parseDoubleQuotesString (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1338:12)
    at parseSymbol (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1237:12)
    at parseCustomNodes (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1210:12)
    at parseLeftHandOperators (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1134:12)
    at parsePow (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1116:12)
    at parseUnary (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1105:12)
    at parsePercentage (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1065:12)
    at parseRule2 (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1016:16)
    at parseImplicitMultiplication (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:988:12)
    at parseMultiplyDivide (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:958:12)
    at parseAddSubtract (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:931:12)
    at parseRange (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:893:14)
    at parseConversion (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:858:12)
    at parseShift (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:835:12)
    at parseRelational (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:797:19)
    at parseBitwiseAnd (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:783:16)
    at parseBitwiseXor (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:768:16)
    at parseBitwiseOr (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:753:16)
    at parseLogicalAnd (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:738:16)
    at parseLogicalXor (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:723:16)
    at parseLogicalOr (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:708:16)
    at parseConditional (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:679:16)
    at parseAssignment (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:632:16)
    at parseBlock (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:580:14)
    at parseStart (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:551:16)
    at string (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:75:14)
    at parse (/src/server/node_modules/typed-function/lib/esm/typed-function.mjs:1790:20)
    at string (/src/server/node_modules/mathjs/lib/cjs/expression/function/evaluate.js:50:14)
    at evaluate (/src/server/node_modules/typed-function/lib/esm/typed-function.mjs:1790:20)
    at /src/server/lib/scene/scene.actions.js:264:17
    at Array.forEach (<anonymous>)
    at Object.condition.only-continue-if (/src/server/lib/scene/scene.actions.js:261:23)
    at executeAction (/src/server/lib/scene/scene.executeActions.js:32:35)
    at /src/server/lib/scene/scene.executeActions.js:66:13
    at tryCatcher (/src/server/node_modules/bluebird/js/release/util.js:16:23)
    at MappingPromiseArray._promiseFulfilled (/src/server/node_modules/bluebird/js/release/map.js:68:38)
    at MappingPromiseArray.PromiseArray._iterate (/src/server/node_modules/bluebird/js/release/promise_array.js:115:31)
    at MappingPromiseArray.init (/src/server/node_modules/bluebird/js/release/promise_array.js:79:10)
    at MappingPromiseArray._asyncInit (/src/server/node_modules/bluebird/js/release/map.js:37:10)
    at _drainQueueStep (/src/server/node_modules/bluebird/js/release/async.js:97:12)
    at _drainQueue (/src/server/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/src/server/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues (/src/server/node_modules/bluebird/js/release/async.js:15:14)
    at processImmediate (node:internal/timers:476:21) {
  char: 1
}
2024-06-23T21:40:51+0200 <warn> scene.executeActions.js:37 (executeAction) Error: Undefined symbol Clouds
    at Function.onUndefinedSymbol (/src/server/node_modules/mathjs/lib/cjs/expression/node/SymbolNode.js:222:15)
    at /src/server/node_modules/mathjs/lib/cjs/expression/node/SymbolNode.js:96:99
    at Object.evaluate (/src/server/node_modules/mathjs/lib/cjs/expression/node/Node.js:77:18)
    at string (/src/server/node_modules/mathjs/lib/cjs/expression/function/evaluate.js:50:36)
    at evaluate (/src/server/node_modules/typed-function/lib/esm/typed-function.mjs:1790:20)
    at Object.device.set-value (/src/server/lib/scene/scene.actions.js:57:15)
    at executeAction (/src/server/lib/scene/scene.executeActions.js:32:35)
    at /src/server/lib/scene/scene.executeActions.js:66:13
    at tryCatcher (/src/server/node_modules/bluebird/js/release/util.js:16:23)
    at MappingPromiseArray._promiseFulfilled (/src/server/node_modules/bluebird/js/release/map.js:68:38)
    at MappingPromiseArray.PromiseArray._iterate (/src/server/node_modules/bluebird/js/release/promise_array.js:115:31)
    at MappingPromiseArray.init (/src/server/node_modules/bluebird/js/release/promise_array.js:79:10)
    at MappingPromiseArray._asyncInit (/src/server/node_modules/bluebird/js/release/map.js:37:10)
    at _drainQueueStep (/src/server/node_modules/bluebird/js/release/async.js:97:12)
    at _drainQueue (/src/server/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/src/server/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues (/src/server/node_modules/bluebird/js/release/async.js:15:14)
    at processImmediate (node:internal/timers:476:21)
2024-06-23T21:40:54+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:41:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:41:04+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:41:13+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:41:22+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:41:32+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:41:41+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:41:51+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:42:00+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:42:02+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
pi@domotique:~ $ docker logs gladys --since 10m | grep SQLITE_BUSY

et « … grep error » returns:

pi@domotique:~ $ docker logs gladys --since 10m | grep error
2024-06-23T21:34:34+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:34:44+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:34:53+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:35:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:35:03+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:35:12+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:35:22+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:35:31+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:35:41+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:35:50+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:35:50+0200<warn> scene.executeActions.js:37 (executeAction) SyntaxError: Value expected (char 1)
    at createSyntaxError (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1623:17)
    at parseEnd (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1590:13)
    at parseParentheses (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1577:12)
    at parseNumber (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1550:12)
    at parseObject (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1534:12)
    at parseMatrix (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1466:12)
    at parseSingleQuotesString (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1382:12)
    at parseDoubleQuotesString (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1338:12)
    at parseSymbol (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1237:12)
    at parseCustomNodes (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1210:12)
    at parseLeftHandOperators (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1134:12)
    at parsePow (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1116:12)
    at parseUnary (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1105:12)
    at parsePercentage (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1065:12)
    at parseRule2 (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1016:16)
    at parseImplicitMultiplication (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:988:12)
    at parseMultiplyDivide (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:958:12)
    at parseAddSubtract (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:931:12)
    at parseRange (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:893:14)
    at parseConversion (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:858:12)
    at parseShift (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:835:12)
    at parseRelational (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:797:19)
    at parseBitwiseAnd (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:783:16)
    at parseBitwiseXor (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:768:16)
    at parseBitwiseOr (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:753:16)
    at parseLogicalAnd (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:738:16)
    at parseLogicalXor (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:723:16)
    at parseLogicalOr (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:708:16)
    at parseConditional (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:679:16)
    at parseAssignment (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:632:16)
    at parseBlock (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:580:14)
    at parseStart (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:551:16)
    at string (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:75:14)
    at parse (/src/server/node_modules/typed-function/lib/esm/typed-function.mjs:1790:20)
    at string (/src/server/node_modules/mathjs/lib/cjs/expression/function/evaluate.js:50:14)
    at evaluate (/src/server/node_modules/typed-function/lib/esm/typed-function.mjs:1790:20)
    at /src/server/lib/scene/scene.actions.js:264:17
    at Array.forEach (<anonymous>)
    at Object.condition.only-continue-if (/src/server/lib/scene/scene.actions.js:261:23)
    at executeAction (/src/server/lib/scene/scene.executeActions.js:32:35)
    at /src/server/lib/scene/scene.executeActions.js:66:13
    at tryCatcher (/src/server/node_modules/bluebird/js/release/util.js:16:23)
    at MappingPromiseArray._promiseFulfilled (/src/server/node_modules/bluebird/js/release/map.js:68:38)
    at MappingPromiseArray.PromiseArray._iterate (/src/server/node_modules/bluebird/js/release/promise_array.js:115:31)
    at MappingPromiseArray.init (/src/server/node_modules/bluebird/js/release/promise_array.js:79:10)
    at MappingPromiseArray._asyncInit (/src/server/node_modules/bluebird/js/release/map.js:37:10)
    at _drainQueueStep (/src/server/node_modules/bluebird/js/release/async.js:97:12)
    at _drainQueue (/src/server/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/src/server/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues (/src/server/node_modules/bluebird/js/release/async.js:15:14)
    at processImmediate (node:internal/timers:476:21) {
  char: 1
}
2024-06-23T21:35:51+0200 <warn> scene.executeActions.js:37 (executeAction) Error: Undefined symbol Clouds
    at Function.onUndefinedSymbol (/src/server/node_modules/mathjs/lib/cjs/expression/node/SymbolNode.js:222:15)
    at /src/server/node_modules/mathjs/lib/cjs/expression/node/SymbolNode.js:96:99
    at Object.evaluate (/src/server/node_modules/mathjs/lib/cjs/expression/node/Node.js:77:18)
    at string (/src/server/node_modules/mathjs/lib/cjs/expression/function/evaluate.js:50:36)
    at evaluate (/src/server/node_modules/typed-function/lib/esm/typed-function.mjs:1790:20)
    at Object.device.set-value (/src/server/lib/scene/scene.actions.js:57:15)
    at executeAction (/src/server/lib/scene/scene.executeActions.js:32:35)
    at /src/server/lib/scene/scene.executeActions.js:66:13
    at tryCatcher (/src/server/node_modules/bluebird/js/release/util.js:16:23)
    at MappingPromiseArray._promiseFulfilled (/src/server/node_modules/bluebird/js/release/map.js:68:38)
    at MappingPromiseArray.PromiseArray._iterate (/src/server/node_modules/bluebird/js/release/promise_array.js:115:31)
    at MappingPromiseArray.init (/src/server/node_modules/bluebird/js/release/promise_array.js:79:10)
    at MappingPromiseArray._asyncInit (/src/server/node_modules/bluebird/js/release/map.js:37:10)
    at _drainQueueStep (/src/server/node_modules/bluebird/js/release/async.js:97:12)
    at _drainQueue (/src/server/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/src/server/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues (/src/server/node_modules/bluebird/js/release/async.js:15:14)
    at processImmediate (node:internal/timers:476:21)
2024-06-23T21:36:00+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:36:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:36:09+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:36:19+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:36:28+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:36:38+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:36:47+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:36:57+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:37:03+0200<warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:37:06+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:37:16+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:37:25+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:37:35+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:37:44+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:37:54+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:38:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:38:03+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:38:12+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:38:22+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:38:32+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:38:41+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:38:51+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:39:00+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:39:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:39:10+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:39:19+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:39:29+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:39:38+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:39:48+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:39:57+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:40:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:40:07+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:40:16+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:40:26+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:40:35+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:40:45+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:40:50+0200 <warn> scene.executeActions.js:37 (executeAction) SyntaxError: Value expected (char 1)
    at createSyntaxError (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1623:17)
    at parseEnd (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1590:13)
    at parseParentheses (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1577:12)
    at parseNumber (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1550:12)
    at parseObject (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1534:12)
    at parseMatrix (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1466:12)
    at parseSingleQuotesString (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1382:12)
    at parseDoubleQuotesString (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1338:12)
    at parseSymbol (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1237:12)
    at parseCustomNodes (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1210:12)
    at parseLeftHandOperators (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1134:12)
    at parsePow (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1116:12)
    at parseUnary (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1105:12)
    at parsePercentage (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1065:12)
    at parseRule2 (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:1016:16)
    at parseImplicitMultiplication (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:988:12)
    at parseMultiplyDivide (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:958:12)
    at parseAddSubtract (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:931:12)
    at parseRange (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:893:14)
    at parseConversion (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:858:12)
    at parseShift (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:835:12)
    at parseRelational (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:797:19)
    at parseBitwiseAnd (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:783:16)
    at parseBitwiseXor (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:768:16)
    at parseBitwiseOr (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:753:16)
    at parseLogicalAnd (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:738:16)
    at parseLogicalXor (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:723:16)
    at parseLogicalOr (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:708:16)
    at parseConditional (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:679:16)
    at parseAssignment (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:632:16)
    at parseBlock (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:580:14)
    at parseStart (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:551:16)
    at string (/src/server/node_modules/mathjs/lib/cjs/expression/parse.js:75:14)
    at parse (/src/server/node_modules/typed-function/lib/esm/typed-function.mjs:1790:20)
    at string (/src/server/node_modules/mathjs/lib/cjs/expression/function/evaluate.js:50:14)
    at evaluate (/src/server/node_modules/typed-function/lib/esm/typed-function.mjs:1790:20)
    at /src/server/lib/scene/scene.actions.js:264:17
    at Array.forEach (<anonymous>)
    at Object.condition.only-continue-if (/src/server/lib/scene/scene.actions.js:261:23)
    at executeAction (/src/server/lib/scene/scene.executeActions.js:32:35)
    at /src/server/lib/scene/scene.executeActions.js:66:13
    at tryCatcher (/src/server/node_modules/bluebird/js/release/util.js:16:23)
    at MappingPromiseArray._promiseFulfilled (/src/server/node_modules/bluebird/js/release/map.js:68:38)
    at MappingPromiseArray.PromiseArray._iterate (/src/server/node_modules/bluebird/js/release/promise_array.js:115:31)
    at MappingPromiseArray.init (/src/server/node_modules/bluebird/js/release/promise_array.js:79:10)
    at MappingPromiseArray._asyncInit (/src/server/node_modules/bluebird/js/release/map.js:37:10)
    at _drainQueueStep (/src/server/node_modules/bluebird/js/release/async.js:97:12)
    at _drainQueue (/src/server/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/src/server/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues (/src/server/node_modules/bluebird/js/release/async.js:15:14)
    at processImmediate (node:internal/timers:476:21) {
  char: 1
}
2024-06-23T21:40:51+0200 <warn> scene.executeActions.js:37 (executeAction) Error: Undefined symbol Clouds
    at Function.onUndefinedSymbol (/src/server/node_modules/mathjs/lib/cjs/expression/node/SymbolNode.js:222:15)
    at /src/server/node_modules/mathjs/lib/cjs/expression/node/SymbolNode.js:96:99
    at Object.evaluate (/src/server/node_modules/mathjs/lib/cjs/expression/node/Node.js:77:18)
    at string (/src/server/node_modules/mathjs/lib/cjs/expression/function/evaluate.js:50:36)
    at evaluate (/src/server/node_modules/typed-function/lib/esm/typed-function.mjs:1790:20)
    at Object.device.set-value (/src/server/lib/scene/scene.actions.js:57:15)
    at executeAction (/src/server/lib/scene/scene.executeActions.js:32:35)
    at /src/server/lib/scene/scene.executeActions.js:66:13
    at tryCatcher (/src/server/node_modules/bluebird/js/release/util.js:16:23)
    at MappingPromiseArray._promiseFulfilled (/src/server/node_modules/bluebird/js/release/map.js:68:38)
    at MappingPromiseArray.PromiseArray._iterate (/src/server/node_modules/bluebird/js/release/promise_array.js:115:31)
    at MappingPromiseArray.init (/src/server/node_modules/bluebird/js/release/promise_array.js:79:10)
    at MappingPromiseArray._asyncInit (/src/server/node_modules/bluebird/js/release/map.js:37:10)
    at _drainQueueStep (/src/server/node_modules/bluebird/js/release/async.js:97:12)
    at _drainQueue (/src/server/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/src/server/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues (/src/server/node_modules/bluebird/js/release/async.js:15:14)
    at processImmediate (node:internal/timers:476:21)
2024-06-23T21:40:54+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:41:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:41:04+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:41:13+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:41:22+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:41:32+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:41:41+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:41:51+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:42:00+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:42:02+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:42:10+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:42:19+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:42:29+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:42:38+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:42:48+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:42:57+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:43:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:43:07+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:43:16+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:43:26+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:43:35+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:43:45+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:43:54+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:44:03+0200 <warn> poll.js:14 (RtspCameraHandler.poll) Unable to poll camera
2024-06-23T21:44:04+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:44:13+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:44:23+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
2024-06-23T21:44:32+0200 <warn> message.connect.js:19 (TelegramBot.<anonymous>) Telegram polling error, code = ETELEGRAM, message = ETELEGRAM: 409 Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
pi@domotique:~ $

If I’m not mistaken, very similar content, with therefore 4 types of errors:

Telegram polling error, every 10 seconds
Unable to poll camera, every minute
SyntaxError: Value expected (char 1), in a scene, every 5 minutes
Error: Undefined symbol Clouds, in a scene, every 5 minutes

For the camera, I indeed have one that changed its IP a few days ago, so I will correct that…

For Telegram, could this be due to the fact that I configured the same bot in gladys and in node-red? I find it convenient that both tools write to the same bot… But if it’s not recommended, I could separate them…

For the two scenes, how can I know which ones they are? And what could these errors mean?

Terdious · June 24, 2024, 5:00am

Hi @StephaneB,

By any chance, have you checked your Gladys Plus backups? If you don’t have anything else on it, and you have 45% of disk space used, Gladys needs at least the same amount to perform the backup.

I know you say it doesn’t necessarily happen at the same time, but you never know ^^
How big is your DB?

pierre-gilles · June 24, 2024, 7:09am

Hi @StephaneB

Indeed it’s unfortunate if Gladys freezes, we’ll find it

Indeed, for that you need 2 API keys. I don’t know if it’s possible to create 2 API keys for the same bot, but in any case you can’t have 2 connections at the same time, it’s a limitation on the Telegram API side.

Apparently it’s a scene with a « continue only if » with a calculation, does that ring a bell?

That comes from a scene with a « Control a device » with a calculated value, does that sound familiar?

Another question: does this blockage occur on the hour? Every hour, Gladys computes the aggregated values of sensors; that could be what’s saturating your disk bandwidth and preventing the rest of Gladys from running

PS: As soon as you find the offending scenes, if that’s indeed what’s causing the blockage, please document here what you found in those scenes so we can fix in Gladys what causes these blockages

StephaneB · June 24, 2024, 8:52pm

Thanks for the tip, @Terdious. But no worries about my backups, they are being done correctly. And the size of the backup is about 5 GB.

StephaneB · June 24, 2024, 8:58pm

Thanks @pierre-gilles for these pointers. I’ll take care of fixing my config with Telegram.

And regarding the scenes, what’s complicated is that I have tons of scenes that match the criteria you gave. Is there a log of each scene trigger with the scene name, so I can cross-reference it with the time of the error?

Otherwise I’ll open them one by one and check which ones run every 5 minutes, and among them those that match your criteria.

StephaneB · June 24, 2024, 9:00pm

And I’m not really sure whether the lockups occur at exact times, because I notice them when I need Gladys and it doesn’t respond, and those moments are fairly random…

pierre-gilles · June 25, 2024, 7:18am

[quote=« StephaneB, post:7, topic:8981 »]
Is there a record of each scene trigger with the scene name, and I’ll cross-check it with the time

StephaneB · June 28, 2024, 11:06pm

Thanks @pierre-gilles for your offer of help. But I’m the one who doesn’t have the time to sit down and call you. I’ll try as soon as possible…

StephaneB · July 1, 2024, 6:00pm

@pierre-gilles, I’ve had 3 crashes in the last few days :

1st time: after a little over 47h of operation, around 6:30
2nd time: after a little over 9h of operation, around 19:00
3rd time: after a little over 2h of operation, around 2:30
and now it’s been working for 12 hours…

In the docker logs, I still have the same two errors already described above… So I started rereading my scenes one by one to find those that trigger every 5 minutes and contain the actions you mentioned… I’ll end up finding them

But I spotted another thing on the ‹ background tasks › page in Gladys: the following error is displayed every hour with the monthly aggregation:

Error: 
\u003c--- Last few GCs ---\u003e
rt of marking 1752 ms) (average mu = 0.672, current mu = 0.238[110976:0x7fb3c20040]  1698423 ms: Mark-sweep (reduce) 919.1 (936.1) -\u003e 919.1 (936.1) MB, 1709.2 / 0.0 ms  (+ 10.0 ms in 3 steps since start of marking, biggest step 3.5 ms, walltime since start of marking 1728 ms) (average mu = 0.532, current mu = 0.214)[110976:0x7fb3c20040]  1700597 ms: Mark-sweep (reduce) 938.5 (955.5) -\u003e 938.3 (955.5) MB, 2126.9 / 0.0 ms  (average mu = 0.339, current mu = 0.022) allocation failure; scavenge might not succeed


\u003c--- JS stacktrace ---\u003e

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

And this error is also, logically, in the docker logs…

Do you know what I should do to eliminate this aggregation error?

pierre-gilles · July 1, 2024, 6:40pm

The aggregation is getting tripped up by a RAM usage issue — Node.js’s « Heap memory » is full; several possible causes:

either your Pi has no RAM available at the moment the aggregation runs (in your error message, the aggregation was using 950MB of RAM)
How are you doing in terms of RAM usage?
or the Heap Memory limit is reached. Out of curiosity, are you on a 32-bit or 64-bit system?

In general, the longer it’s been since an aggregation ran, the worse it gets, because there are more and more states to « process » and so it becomes increasingly complicated ^^

StephaneB · July 1, 2024, 6:55pm

RAM smoothing oscillates between 5% and 30%, according to this diagram :

And the spike to 55% corresponds to the last crash

StephaneB · July 1, 2024, 6:58pm

And I’d say it’s 64-bit, but I don’t remember how to check

GBoulvin · July 1, 2024, 6:59pm

Attention, we’re talking about RAM, volatile memory, not CPU load…

GBoulvin · July 1, 2024, 7:01pm

When connecting via SSH, first line

Aarch64, 64-bit

StephaneB · July 1, 2024, 7:08pm

Oops… Indeed, I wasn’t attentive enough to respond.

Regarding RAM, here’s what the piHelper app shows me on my smartphone:

Will_71 · July 1, 2024, 7:09pm

With the commands uname -m or arch you’ll get your version

StephaneB · July 1, 2024, 7:36pm

Thank you. So it’s ‹ aarch64 ›.

pierre-gilles · July 1, 2024, 7:36pm

That doesn’t help us much, it’s a figure relative to a single point in time; I’d like to know how much RAM you have available and how much is generally in use.

On your Pi, from the command line you can run:

docker stats

Or:

htop

Take screenshots of both commands for us

Topic		Replies	Views
Configuration MQTT : Erreur lors de la connexion, veuillez vérifier votre configuration. Après restauration de Gladys sur l'image Bullseye Configuration	31	1575	December 16, 2021
Problème activation du service Zigbee2mqtt Configuration	33	3364	February 20, 2022
Service ewelink en erreur Configuration	58	1867	June 29, 2024
Help! Problème avec l'agrégation? Configuration	89	3674	March 29, 2022
Problème de RAM - Gladys ne réagit plus Configuration	10	604	March 21, 2023

Gladys becomes unreachable after a few hours of operation

Related topics