Problem connecting to MQTT after mosquit

_Will_71 · December 16, 2024, 11:53am

I confirm that I have already had at least 3 similar disconnections on Z-Wave JS (zwavejs) since its integration.
Each time I had to go to the configuration page and save, and then it worked again.
But not recently.

pierre-gilles · December 16, 2024, 7:27pm

Thanks for your feedback

@_Will_71 Do you think you could investigate this the next time it happens to you?

On my end, despite all my efforts, I can’t reproduce it… I’d like to help, it’s frustrating to see these reports

_Will_71 · December 16, 2024, 7:43pm

I’ll try, but

gaetanb76 · January 11, 2025, 10:13pm

I just « forcefully » restarted the mini PC because I no longer had access to it (powerline adapter (CPL) issue).
The MQTT broker had not restarted, but the Z-Wave JS UI one had.

pierre-gilles · January 13, 2025, 8:57am

I’m reopening the discussion here following @mutmut’s message on another thread

Regarding Gladys reconnecting to the MQTT broker after a lost connection / a broker restart, this is normally entirely handled by the library we use. Reconnection is automatic!

There must be a particular case happening on your side that puts the library in a messed-up state. I tried to reproduce everything that was described to me, but on my setup the reconnection works fine. I can’t reproduce it, it’s very frustrating ^^

@Terdious is the only developer who managed to reproduce the bug on his machine but he’s busy with insulation work and hasn’t had time to look yet.

If anyone wants to help us, we’d really appreciate help with this issue!

Terdious · January 14, 2025, 6:52am

Hello !!

I haven’t finished the work yet but I have what I need to reinstall my dev machine.

I’ll look into it as soon as possible (this weekend at the latest) to at least be able to see what’s wrong and guide us.

_Will_71 · January 14, 2025, 12:51pm

I haven’t been able to reproduce it on my end so far.

Terdious · January 15, 2025, 6:02am

1er logs, au redémarrage de gladys :

2025-01-15T06:49:16+0100 <error> index.js:15 (process.<anonymous>) unhandledRejection catched: Promise {
  <rejected> NotFoundError: DeviceFeature tasmota:tasmota_836534:ENERGY:Voltage not found
      at DeviceManager.newStateEvent (/home/terdieu/PR-Gladys/netatmo-integration/server/lib/device/device.newStateEvent.js:17:11)
      at EventEmitter.emit (node:events:517:28)
      at Event.emit (/home/terdieu/PR-Gladys/netatmo-integration/server/lib/event/index.js:18:16)
}
2025-01-15T06:49:16+0100 <error> index.js:16 (process.<anonymous>) 
2025-01-15T06:49:16+0100 <error> index.js:15 (process.<anonymous>) unhandledRejection catched: Promise {
  <rejected> NotFoundError: DeviceFeature tasmota:tasmota_836534:ENERGY:Current not found
      at DeviceManager.newStateEvent (/home/terdieu/PR-Gladys/netatmo-integration/server/lib/device/device.newStateEvent.js:17:11)
      at EventEmitter.emit (node:events:517:28)
      at Event.emit (/home/terdieu/PR-Gladys/netatmo-integration/server/lib/event/index.js:18:16)
}
2025-01-15T06:49:16+0100 <error> index.js:16 (process.<anonymous>) 
2025-01-15T06:49:16+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:49:16+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage) 
2025-01-15T06:49:16+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:49:16+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage) 
2025-01-15T06:49:16+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:49:16+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage) 
2025-01-15T06:49:16+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:49:16+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage) 
2025-01-15T06:49:16+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:49:16+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage) 
2025-01-15T06:49:16+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:49:16+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage)

Il réussit donc bien à se connecter et on trouve environ 650 lignes de logs de message reçus la meme seconde.
Puis :

2025-01-15T06:49:16+0100 <warn> connect.js:50 (MqttClient.<anonymous>) Error while connecting to MQTT - Error: write ECONNRESET

il plante ligne 50 du connect
Si j’ajoute des logs dans la fonction du connect this.mqttClient.on('message', (topic, message) j’obtiens des logs de buffer :

2025-01-15T06:57:50+0100 <error> index.js:16 (process.<anonymous>) 
2025-01-15T06:57:50+0100 <error> index.js:15 (process.<anonymous>) unhandledRejection catched: Promise {
  <rejected> NotFoundError: DeviceFeature tasmota:tasmota_837F73:ENERGY:ApparentPower not found
      at DeviceManager.newStateEvent (/home/terdieu/PR-Gladys/netatmo-integration/server/lib/device/device.newStateEvent.js:17:11)
      at EventEmitter.emit (node:events:517:28)
      at Event.emit (/home/terdieu/PR-Gladys/netatmo-integration/server/lib/event/index.js:18:16)
}
2025-01-15T06:57:50+0100 <error> index.js:16 (process.<anonymous>) 
2025-01-15T06:57:50+0100 <error> index.js:15 (process.<anonymous>) unhandledRejection catched: Promise {
  <rejected> NotFoundError: DeviceFeature tasmota:tasmota_837F73:ENERGY:ReactivePower not found
      at DeviceManager.newStateEvent (/home/terdieu/PR-Gladys/netatmo-integration/server/lib/device/device.newStateEvent.js:17:11)
      at EventEmitter.emit (node:events:517:28)
      at Event.emit (/home/terdieu/PR-Gladys/netatmo-integration/server/lib/event/index.js:18:16)
}
2025-01-15T06:57:50+0100 <error> index.js:16 (process.<anonymous>) 
2025-01-15T06:57:50+0100 <error> index.js:15 (process.<anonymous>) unhandledRejection catched: Promise {
  <rejected> NotFoundError: DeviceFeature tasmota:tasmota_837F73:ENERGY:Voltage not found
      at DeviceManager.newStateEvent (/home/terdieu/PR-Gladys/netatmo-integration/server/lib/device/device.newStateEvent.js:17:11)
      at EventEmitter.emit (node:events:517:28)
      at Event.emit (/home/terdieu/PR-Gladys/netatmo-integration/server/lib/event/index.js:18:16)
}
2025-01-15T06:57:50+0100 <error> index.js:16 (process.<anonymous>) 
2025-01-15T06:57:50+0100 <error> index.js:15 (process.<anonymous>) unhandledRejection catched: Promise {
  <rejected> NotFoundError: DeviceFeature tasmota:tasmota_837F73:ENERGY:Current not found
      at DeviceManager.newStateEvent (/home/terdieu/PR-Gladys/netatmo-integration/server/lib/device/device.newStateEvent.js:17:11)
      at EventEmitter.emit (node:events:517:28)
      at Event.emit (/home/terdieu/PR-Gladys/netatmo-integration/server/lib/event/index.js:18:16)
}
2025-01-15T06:57:50+0100 <error> index.js:16 (process.<anonymous>) 
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 54 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 55 70 74 69 6d 65 ... 302 more bytes>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 36 22 7d 7d>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 54 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 55 70 74 69 6d 65 ... 267 more bytes>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 54 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 55 70 74 69 6d 65 ... 267 more bytes>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 53 77 69 74 63 68 ... 10 more bytes>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 54 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 55 70 74 69 6d 65 ... 271 more bytes>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 54 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 55 70 74 69 6d 65 ... 306 more bytes>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 7d 7d>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 45 4e 45 52 47 59 ... 184 more bytes>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 7d 7d>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 36 22 7d 7d>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 54 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 55 70 74 69 6d 65 ... 271 more bytes>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 54 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 36 22 2c 22 55 70 74 69 6d 65 ... 300 more bytes>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 7d 7d>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 53 77 69 74 63 68 ... 10 more bytes>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 45 4e 45 52 47 59 ... 182 more bytes>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 36 22 7d 7d>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 54 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 55 70 74 69 6d 65 ... 302 more bytes>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 45 4e 45 52 47 59 ... 180 more bytes>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 36 22 7d 7d>
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 45 4e 45 52 47 59 ... 178 more bytes>
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 45 4e 45 52 47 59 ... 178 more bytes>
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 45 4e 45 52 47 59 ... 178 more bytes>
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 45 4e 45 52 47 59 ... 184 more bytes>
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 2c 22 45 4e 45 52 47 59 ... 184 more bytes>
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 7d 7d>
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 37 22 7d 7d>
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 54 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 36 22 2c 22 55 70 74 69 6d 65 ... 300 more bytes>
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 36 22 7d 7d>
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <error> connect.js:68 (MqttClient.<anonymous>) message <Buffer 7b 22 53 74 61 74 75 73 53 4e 53 22 3a 7b 22 54 69 6d 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 35 54 30 36 3a 35 37 3a 34 36 22 7d 7d>
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:53 (MqttHandler.handleNewMessage) 
2025-01-15T06:57:50+0100 <warn> handleNewMessage.js:54 (MqttHandler.handleNewMessage)

Bon après l’erreur est assez clair : Error: write ECONNRESET

EDIT:
Bon … je ne peux plus poursuivre les tests pour le moment, visiblement j’ai pété DuckDB avec toutes ces écritures :

> start
> cross-env NODE_ENV=development nodemon index.js

[nodemon] 2.0.20
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node index.js`
development
2025-01-15T07:12:43+0100 <error> index.js:15 (process.<anonymous>) unhandledRejection catched: Promise {
  <rejected> [Error: Connection Error: Connection was never established or has been closed already] {
    errno: -1,
    code: 'DUCKDB_NODEJS_ERROR',
    errorType: 'Connection'
  }
}
2025-01-15T07:12:43+0100 <error> index.js:16 (process.<anonymous>) [Error: Connection Error: Connection was never established or has been closed already] {
  errno: -1,
  code: 'DUCKDB_NODEJS_ERROR',
  errorType: 'Connection'
}
[nodemon] clean exit - waiting for changes before restart

Terdious · January 15, 2025, 7:19am

I managed to reconnect to the DB.

@pierre-gilles
If in the file server/services/mqtt/lib/connect.js I add a single automatic reconnection in the mqttClient.on 'error':

  this.mqttClient.on('error', (err) =\u003e {
    logger.warn(`Error while connecting to MQTT - ${err}`);

    this.gladys.event.emit(EVENTS.WEBSOCKET.SEND_ALL, {
      type: WEBSOCKET_MESSAGE_TYPES.MQTT.ERROR,
      payload: err,
    });

    this.disconnect();
  });

or

  this.mqttClient.on('error', (err) =\u003e {
    logger.warn(`Error while connecting to MQTT - ${err}`);

    this.gladys.event.emit(EVENTS.WEBSOCKET.SEND_ALL, {
      type: WEBSOCKET_MESSAGE_TYPES.MQTT.ERROR,
      payload: err,
    });

    if (!this.reconnecting) {
      this.reconnecting = true;
      logger.warn('Attempting to reconnect to MQTT server...');

      setTimeout(() =\u003e {
        this.connect({
          mqttUrl: mqttUrl,
          mqttUsername: mqttUsername,
          mqttPassword: mqttPassword
        }).finally(() =\u003e {
          this.reconnecting = false;
        });
      }, 500);
    } else {
      this.disconnect();
    }

After 500ms I’m successfully reconnected automatically.
So I suppose the library’s automatic reconnection doesn’t work with the external broker? And that what we see in the buffer during the connection attempt are just the previous logs.
This may not be the right method. Tell me again.

pierre-gilles · January 16, 2025, 7:55am

@Terdious that’s one of the possibilities, yes

As I said above, for me there’s also testing the library update. We’re using an old version, and maybe automatic reconnection has been fixed in a more recent version

Terdious · January 16, 2025, 9:18am

Hello @pierre-gilles,

I just tried. No change! I reverted my edit and updated to 4.2.6 (you’ll tell me that doesn’t prevent the update ^^).

Apparently it happens with an internal broker (localhost) as well as an external one.
And on the internal container side in gladys we’re using the same fixed version and apparently @gaetanb76 is encountering the same issue…

My impression was that on connection, too much data was received and that it messed things up. But in this case there would be no reason for @gaetanb76 to be in the same situation since the broker restarts at the same time.

But the piece of code for automatic reconnection does work for him … ^^

Terdious · January 16, 2025, 9:50am

All right I found the cause of the problem

I’ll see if I can fix it!!^^

I had thought that the initial problem had been fixed at one point, then the issue came back ^^

In server/services/mqtt/lib/listenToCustomMqttTopicIfNeeded.js if I remove in the code:

    if (this.mqttClient) {
      logger.info(`Subscribing to MQTT topic ${mqttTopic}`);
      // this.mqttClient.subscribe(mqttTopic);
    }

Then no more connection issues on restart. If I do:

    if (this.mqttClient) {
      logger.error(`Subscribing to MQTT topic ${mqttTopic}`);
      // this.mqttClient.subscribe(mqttTopic);
      try {
        const result = this.mqttClient.subscribe(mqttTopic);
        logger.warn(result);
      } catch (error) {
        logger.error(`Error subscribing to MQTT topic ${mqttTopic}: ${error}`);

      }
    }

The logs:

2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic 
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic : RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic 
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic : RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic gladys/master/device/mqtt:light-ambiance-piscine/feature/mqtt:light-ambiance-piscine
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic gladys/master/device/mqtt:light-ambiance-piscine/feature/mqtt:light-ambiance-piscine: RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic gladys/master/device/mqtt:light-ambiance-piscine/feature/mqtt:light-ambiance-piscine
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic gladys/master/device/mqtt:light-ambiance-piscine/feature/mqtt:light-ambiance-piscine: RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic tarifs-total-energie
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic tarifs-total-energie: RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic tarifs-total-energie
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic tarifs-total-energie: RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic consos-vince
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic consos-vince: RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic consos-vince
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic consos-vince: RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic consos-vince
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic consos-vince: RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic consos-vince
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic consos-vince: RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic consos-vince
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic consos-vince: RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic 
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic : RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic consos-vince
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic consos-vince: RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic consos-vince
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic consos-vince: RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic consos-vince
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic consos-vince: RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:32 () Subscribing to MQTT topic consos-vince
2025-01-16T11:05:17+0100 <error> listenToCustomMqttTopicIfNeeded.js:38 () Error subscribing to MQTT topic consos-vince: RangeError: Maximum call stack size exceeded
2025-01-16T11:05:17+0100 <warn> connect.js:54 (MqttClient.<anonymous>) Error while connecting to MQTT - Error: write ECONNRESET
2025-01-16T11:05:17+0100 <warn> connect.js:80 (MqttClient.<anonymous>) Disconnected from MQTT server
2025-01-16T11:05:22+0100 <error> connect.js:41 (MqttClient.<anonymous>) Connected to MQTT server mqtt://10.5.0.50:1883
2025-01-16T11:05:22+0100 <warn> connect.js:80 (MqttClient.<anonymous>) Disconnected from MQTT server

So it no longer wants to reconnect… I don’t know why…
And pressing the save button doesn’t trigger any logs from this part of the code (so no reconnection to the custom topics??)

pierre-gilles · January 16, 2025, 6:42pm

@Terdious that’s so cool if you found it!

Do you use the custom MQTT topic feature a lot?
Do you have multiple devices listening to the same MQTT topic?

Terdious · January 16, 2025, 7:15pm

Yes, still ^^

Yes, several devices but only for one topic. There, apparently no custom topic is getting through.

pierre-gilles · January 16, 2025, 7:23pm

And you put custom topics into Gladys’ native topics? it’s not meant for that

Otherwise, we need to see whether using the same topic multiple times across different features is what’s causing the problem.

Terdious · January 17, 2025, 9:38am

Hi @pierre-gilles,

Following your reply, I pushed the logs a bit further. It’s clearly an error in the fields… So the issue will probably be how to alert about this but not only that. Let me explain: in the custom file I logged:

device.param :

device.params [
  {
    id: 'df3028da-f845-43a4-8f16-2ee02aa15b39',
    device_id: '088f3819-2f5e-4622-8f5b-b3e3b7710918',
    name: 'mqtt_custom_topic_feature:c2ffe676-bd49-4009-a193-ca5b2d6bcf2a',
    value: '',
    created_at: 2024-07-04T04:09:00.529Z,
    updated_at: 2024-07-04T04:35:56.381Z
  },
  {
    id: '1f3089f0-7179-4ad7-b133-79ad2d82a7c5',
    device_id: '088f3819-2f5e-4622-8f5b-b3e3b7710918',
    name: 'mqtt_custom_object_path_feature:c2ffe676-bd49-4009-a193-ca5b2d6bcf2a',
    value: '',
    created_at: 2024-07-04T04:09:00.529Z,
    updated_at: 2024-07-04T04:35:56.386Z
  },
  {
    id: 'abc366a4-247f-496c-bd0a-a0ee9629c43d',
    device_id: '088f3819-2f5e-4622-8f5b-b3e3b7710918',
    name: 'mqtt_custom_topic_feature:b47df305-53f7-4d47-9610-1b630bd49ac7',
    value: '',
    created_at: 2024-07-04T04:09:00.529Z,
    updated_at: 2024-07-04T04:35:56.385Z
  },
  {
    id: '6e31226e-6b03-4908-870e-687403b05fbb',
    device_id: '088f3819-2f5e-4622-8f5b-b3e3b7710918',
    name: 'mqtt_custom_object_path_feature:b47df305-53f7-4d47-9610-1b630bd49ac7',
    value: '',
    created_at: 2024-07-04T04:09:00.529Z,
    updated_at: 2024-07-04T04:35:56.387Z
  },
  {
    id: '500b2c1f-d6ee-4e66-84b7-2612ec3c2414',
    device_id: '088f3819-2f5e-4622-8f5b-b3e3b7710918',
    name: 'mqtt_custom_topic_feature:62b057da-d367-4972-97d3-00dc8fc1a065',
    value: 'gladys/master/device/mqtt:light-ambiance-piscine/feature/mqtt:light-ambiance-piscine',
    created_at: 2024-07-04T04:15:43.875Z,
    updated_at: 2024-07-04T04:15:43.875Z
  },
  {
    id: 'da0d94b8-79bf-4ffe-9058-c997a3debc1d',
    device_id: '088f3819-2f5e-4622-8f5b-b3e3b7710918',
    name: 'mqtt_custom_object_path_feature:62b057da-d367-4972-97d3-00dc8fc1a065',
    value: 'speed',
    created_at: 2024-07-04T04:15:43.875Z,
    updated_at: 2024-07-04T04:24:24.834Z
  },

We can see empty 'value’s, ‹ . ›, 'value’s that are the MQTT JSON message rather than the topic, for example:

this.deviceFeatureCustomMqttTopics

this.deviceFeatureCustomMqttTopics [
  {
    topic: '',
    regex_key: '',
    device_feature_id: 'c2ffe676-bd49-4009-a193-ca5b2d6bcf2a',
    object_path: ''
  },
  {
    topic: '',
    regex_key: '',
    device_feature_id: 'b47df305-53f7-4d47-9610-1b630bd49ac7',
    object_path: ''
  },
  {
    topic: 'gladys/master/device/mqtt:light-ambiance-piscine/feature/mqtt:light-ambiance-piscine',
    regex_key: 'gladys/master/device/mqtt:light-ambiance-piscine/feature/mqtt:light-ambiance-piscine',
    device_feature_id: '62b057da-d367-4972-97d3-00dc8fc1a065',
    object_path: 'speed'
  }
]

And already here’s a problem: empty values… and some filled ones that no longer exist. For example this feature no longer exists. But the params are still there (one-day test):

And conversely, in the following example I indeed have an existing feature with parameters where the value is empty…

But then, I didn’t fill anything in the custom fields (and for those I’m sure I never filled anything - however others were populated during creation):

Could there be an issue with params management when deleting features + others?

Terdious · January 17, 2025, 10:04am

And indeed I just tested deleting a feature, namely the feature with id b6929839-c590-460d-abb1-28c795ed5628 — it’s no longer in the features but still in the params:

And indeed, after thinking about it, it doesn’t surprise me since params are attached to devices — the link must not exist.

So if we configure a custom topic the same way, I suppose that if we remove it afterwards, it just gets replaced by an empty field.

@gaetanb76, can you confirm if you deleted any features or custom topics?

So, test of deleting the affected devices then creating new devices with custom: No more issues :

this.deviceFeatureCustomMqttTopics [
  {
    topic: 'test',
    regex_key: 'test',
    device_feature_id: 'd1a558eb-07d6-4972-900d-c3b4bc1587b8',
    object_path: null
  }
]
device.params [
  {
    id: 'b1795c88-c39e-447a-a683-81f8c6fe1e6a',
    device_id: '6cb64637-71b2-4b34-be5c-75a0eeaff17c',
    name: 'mqtt_custom_topic_feature:d1a558eb-07d6-4972-900d-c3b4bc1587b8',
    value: 'test',
    created_at: 2025-01-17T10:09:39.371Z,
    updated_at: 2025-01-17T10:14:24.625Z
  }
]
2025-01-17T11:14:43+0100 <error> listenToCustomMqttTopicIfNeeded.js:35 () Subscribing to MQTT topic test

If we then clear the custom topic, we can clearly see that the problem therefore comes from the deletion or the clearing:

this.deviceFeatureCustomMqttTopics [
  {
    topic: '',
    regex_key: '',
    device_feature_id: 'd1a558eb-07d6-4972-900d-c3b4bc1587b8',
    object_path: null
  }
]
device.params [
  {
    id: 'b1795c88-c39e-447a-a683-81f8c6fe1e6a',
    device_id: '6cb64637-71b2-4b34-be5c-75a0eeaff17c',
    name: 'mqtt_custom_topic_feature:d1a558eb-07d6-4972-900d-c3b4bc1587b8',
    value: '',
    created_at: 2025-01-17T10:09:39.371Z,
    updated_at: 2025-01-17T10:15:27.005Z
  }
]
2025-01-17T11:15:27+0100 <error> listenToCustomMqttTopicIfNeeded.js:35 () Subscribing to MQTT topic 
2025-01-17T11:15:27+0100 <warn> connect.js:82 (MqttClient.<anonymous>) Disconnected from MQTT server

pierre-gilles · January 17, 2025, 10:19am

Good catch @Terdious! That’s definitely it!

I think that can be fixed pretty easily by adding an if(mqttTopic && mqttTopic.length)

It’s not that bad that the params are empty, listenToCustomMqttTopicIfNeeded.js should just ignore them

Terdious · January 17, 2025, 10:21am

I’m updating the PR with

Terdious · January 17, 2025, 10:38am

Looks good to me @pierre-gilles:

github.com/GladysAssistant/Gladys

Fix MQTT auto-reconnect with custom topic issues + update mqtt

master ← Terdious:fix/mqtt-broker-reconnect-after-restart

ouvert 06:28AM - 16 Jan 25 UTC

Terdious

+333 -861

The code changes fix the MQTT reconnect logic in the connect.js file. It now che…cks if the reconnecting flag is set before disconnecting. If the flag is not set, it sets the flag to true, logs a warning message, and attempts to reconnect to the MQTT server after a 500ms delay. Once the reconnection is complete, the flag is set back to false. If the flag is already set, it simply disconnects from the MQTT server. Additionally, the index.js file now initializes the reconnecting flag to false. ### Pull Request check-list To ensure your Pull Request can be accepted as fast as possible, make sure to review and check all of these items: - [x] If your changes affects code, did your write the tests? - [x] Are tests passing? (`npm test` on both front/server) - [x] Is the linter passing? (`npm run eslint` on both front/server) - [x] Did you run prettier? (`npm run prettier` on both front/server) - [x] Did you test this pull request in real life? With real devices? If this development is a big feature or a new service, we recommend that you provide a Docker image to the community ([french forum](https://community.gladysassistant.com/)/[english forum](https://en-community.gladysassistant.com/)) for testing before merging. NOTE: these things are not required to open a PR and can be done afterwards / while the PR is open. ### Description of change Fixed mqtt service auto-reconnect issue which was caused by mishandled custom topic issues

Topic		Replies	Views
Bug v4.17.0 Zigbee2MQTT? Développement	50	1944	April 10, 2023
Configuration MQTT : Erreur lors de la connexion, veuillez vérifier votre configuration. Après restauration de Gladys sur l'image Bullseye Configuration	31	1533	December 16, 2021
ZigbeeMQTT : aucune info remontée sur l'interface Configuration	6	330	June 27, 2023
Problème pour récupérer une valeur TEXT Configuration mqtt	19	169	May 26, 2025
Gladys Assistant 4.2 est disponible avec la compatibilité Zigbee2mqtt ! 🚀 Actualités	146	5617	May 17, 2021

Problem connecting to MQTT after mosquit

Related topics