[BUG] Suppression device Zigbee

[spenceur]

Good evening,
I currently have a blocking issue on Gladys side.

I renamed the friendly name on the zigbee side to better understand which device is which on the schema.

But to my great surprise, on the Gladys side I had no more feedback from my devices. I then took the time to look at the integration side (IHM), and there I realize that they are considered as new devices :o.

Why was the choice made to base it on the friendly name rather than the IEEE address?

Well, at the moment I thought, well never mind, we’ll delete the old devices and re-add the new ones, but the problem is there.
When I delete a z2m device on the Gladys side, everything freezes and I have to restart the RPI for it to return to normal.

When clicking on « delete » (the spinner spins without response):

Network side:

htop:

The logs (nothing shocking):

2022-04-04T20:50:27+0200 <warn> handleMqttMessage.js:100 () Zigbee2mqtt device Prise 1, feature indicator_mode not configured in Gladys.
2022-04-04T20:50:27+0200 <warn> handleMqttMessage.js:100 () Zigbee2mqtt device Prise 1, feature linkquality not configured in Gladys.
2022-04-04T20:50:27+0200 <warn> handleMqttMessage.js:100 () Zigbee2mqtt device Prise 1, feature power_outage_memory not configured in Gladys.
2022-04-04T20:51:00+0200 <info> scene.checkCalendarTriggers.js:24 (SceneManager.checkCalendarTriggers) Checking calendar triggers at Mon, 04 Apr 2022 18:51:00 GMT
2022-04-04T20:52:00+0200 <info> scene.checkCalendarTriggers.js:24 (SceneManager.checkCalendarTriggers) Checking calendar triggers at Mon, 04 Apr 2022 18:52:00 GMT
2022-04-04T20:52:07+0200 <error> handleMqttMessage.js:97 () Failed to convert value for device Prise 2: Error: Zigbee2mqqt don't handle value "UNLOCK" for feature "mode".
    at convertValue (/src/server/services/zigbee2mqtt/utils/convertValue.js:55:17)
    at /src/server/services/zigbee2mqtt/lib/handleMqttMessage.js:93:26
    at Array.forEach (<anonymous>)
    at Zigbee2mqttManager.handleMqttMessage (/src/server/services/zigbee2mqtt/lib/handleMqttMessage.js:86:41)
    at MqttClient.<anonymous> (/src/server/services/zigbee2mqtt/lib/connect.js:53:12)
    at MqttClient.emit (events.js:400:28)
    at MqttClient._handlePublish (/src/server/services/zigbee2mqtt/node_modules/mqtt/lib/client.js:1277:12)
    at MqttClient._handlePacket (/src/server/services/zigbee2mqtt/node_modules/mqtt/lib/client.js:410:12)
    at work (/src/server/services/zigbee2mqtt/node_modules/mqtt/lib/client.js:321:12)
    at processTicksAndRejections (internal/process/task_queues.js:77:11)
2022-04-04T20:52:07+0200 <warn> handleMqttMessage.js:100 () Zigbee2mqtt device Prise 2, feature indicator_mode not configured in Gladys.
2022-04-04T20:52:07+0200 <warn> handleMqttMessage.js:100 () Zigbee2mqtt device Prise 2, feature linkquality not configured in Gladys.
2022-04-04T20:52:07+0200 <warn> handleMqttMessage.js:100 () Zigbee2mqtt device Prise 2, feature power_outage_memory not configured in Gladys.
2022-04-04T20:52:07+0200 <error> handleMqttMessage.js:97 () Failed to convert value for device Prise 2: Error: Zigbee2mqqt don't handle value "UNLOCK" for feature "mode".
    at convertValue (/src/server/services/zigbee2mqtt/utils/convertValue.js:55:17)
    at /src/server/services/zigbee2mqtt/lib/handleMqttMessage.js:93:26
    at Array.forEach (<anonymous>)
    at Zigbee2mqttManager.handleMqttMessage (/src/server/services/zigbee2mqtt/lib/handleMqttMessage.js:86:41)
    at MqttClient.<anonymous> (/src/server/services/zigbee2mqtt/lib/connect.js:53:12)
    at MqttClient.emit (events.js:400:28)
    at MqttClient._handlePublish (/src/server/services/zigbee2mqtt/node_modules/mqtt/lib/client.js:1277:12)
    at MqttClient._handlePacket (/src/server/services/zigbee2mqtt/node_modules/mqtt/lib/client.js:410:12)
    at work (/src/server/services/zigbee2mqtt/node_modules/mqtt/lib/client.js:321:12)
    at processTicksAndRejections (internal/process/task_queues.js:77:11)
2022-04-04T20:52:07+0200 <warn> handleMqttMessage.js:100 () Zigbee2mqtt device Prise 2, feature indicator_mode not configured in Gladys.
2022-04-04T20:52:07+0200 <warn> handleMqttMessage.js:100 () Zigbee2mqtt device Prise 2, feature linkquality not configured in Gladys.
2022-04-04T20:52:07+0200 <warn> handleMqttMessage.js:100 () Zigbee2mqtt device Prise 2, feature power_outage_memory not configured in Gladys.
2022-04-04T20:52:07+0200 <error> handleMqttMessage.js:97 () Failed to convert value for device Prise 2: Error: Zigbee2mqqt don't handle value "UNLOCK" for feature "mode".
    at convertValue (/src/server/services/zigbee2mqtt/utils/convertValue.js:55:17)
    at /src/server/services/zigbee2mqtt/lib/handleMqttMessage.js:93:26
    at Array.forEach (<anonymous>)
    at Zigbee2mqttManager.handleMqttMessage (/src/server/services/zigbee2mqtt/lib/handleMqttMessage.js:86:41)
    at MqttClient.<anonymous> (/src/server/services/zigbee2mqtt/lib/connect.js:53:12)
    at MqttClient.emit (events.js:400:28)
    at MqttClient._handlePublish (/src/server/services/zigbee2mqtt/node_modules/mqtt/lib/client.js:1277:12)
    at MqttClient._handlePacket (/src/server/services/zigbee2mqtt/node_modules/mqtt/lib/client.js:410:12)
    at work (/src/server/services/zigbee2mqtt/node_modules/mqtt/lib/client.js:321:12)
    at processTicksAndRejections (internal/process/task_queues.js:77:11)
2022-04-04T20:52:07+0200 <warn> handleMqttMessage.js:100 () Zigbee2mqtt device Prise 2, feature indicator_mode not configured in Gladys.
2022-04-04T20:52:07+0200 <warn> handleMqttMessage.js:100 () Zigbee2mqtt device Prise 2, feature linkquality not configured in Gladys.
2022-04-04T20:52:07+0200 <warn> handleMqttMessage.js:100 () Zigbee2mqtt device Prise 2, feature power_outage_memory not configured in Gladys.

Gladys after page refresh (it spins for more than an hour without result):

On the Gladys + side:

Any ideas @pierre-gilles & @cicoub13?

Good evening

[pierre-gilles]

I can’t answer your questions about the choices made during the development of the Zigbee2mqtt integration, but I can answer your question about the UI freeze. This is a known and « normal » limitation of the current implementation of device removal.

When you try to delete a device with tens or hundreds of thousands of states, the database struggles a bit to delete everything in cascade at once, and it can take several minutes to delete everything depending on the speed of your disk. During this time, the instance is frozen because the database is busy (all the bandwidth of your disk is saturated by the cascade deletion).

We have a medium-term solution, but it’s a development. I was recently talking about it here on the forum:

[Recherche de testeurs] Nouvelle image Gladys Raspberry Pi OS sous Bullseye

Je sais ce qu’il se passe, tu as du effectivement supprimer un appareil. Sauf qu’actuellement dans Gladys, quand tu supprime un appareil, ça va supprimer en cascade toutes les device_features ainsi que les états de ces features Tu l’as compris, si derrière cet appareil a des milliers d’états, ça peut prendre 2 minutes et pendant ce temps là, la DB est bloquée car c’est une énorme transaction qui prend toute la bande passante du disque. On a pensé à une solution si ce problème devie…

The only thing I can advise you is to just wait

[spenceur]

I can’t clean up the states in the db, can I?

[pierre-gilles]

Yes, it works too!

[pierre-gilles]

When I said “wait”, it wasn’t wait for the feature to be developed, it was click the button and wait the 2 minutes it takes to delete the data

Manually deleting in the DB will be the same, the DB will also be locked

[spenceur]

Thanks for the info

[pierre-gilles]

I’m following up on this topic — I’ve developed a fix for this limitation so it will be clearer in the future

When deleting a device with many states, the user will see this kind of message:

The PR:

Add message to warn user that a device has lots of states before deleting

[gaetanb76]

Hi @pierre-gilles,
Not having used Gladys for a long time, I wanted to check on my ZigBee2MQTT devices.
I deleted them and received the message about the large volume of data when I removed the light sensor.
However, when I deleted another device, the message stayed in its place and thus ended up in the info of another device.

[pierre-gilles]

@gaetanb76

However, when I deleted another device, the message remained in place and thus ended up in the info of another device.

Oops! I can clearly see the bug, thanks for the feedback. It’s just a display issue, I created a GitHub issue:

List of devices don't have a key and sometimes devices are merged when one is deleted