Backup of the zigbee2mqtt database

AlexTrovato · November 10, 2022, 6:45pm

If you’re subscribed to Gladys Plus, as soon as @lmilcent confirms that the z2m config backup is working, you’ll be able to change machines without worry.

lmilcent · November 10, 2022, 7:19pm

Yes, I recommend waiting for the migration to be handled by Gladys Plus — it will be more turnkey.

Also, I spent several hours because I did some hardening (security hardening) on my server, which caused a few errors.

pierre-gilles · November 11, 2022, 5:07am

@lmilcent Let us know if it works then! No need to test with Gladys Plus by the way, you can just copy the SQLite DB from one system to another and see if it works.

If you ever start another instance in parallel with your prod, be careful because on the Gladys Plus side only one instance can be connected at a time, the last one connected « wins » over the other. So if you ever have two instances running at the same time, once only one remains, restart your instance and it will take over.

I hope you’ll make a little post for us!! That setup is dreamy!

guim31 · November 11, 2022, 7:20am

Thank you for your answers! I’ll wait a little longer then

lmilcent · November 11, 2022, 2:02pm

Voici ce que je viens de faire pour tester.

Clone de ma DB actuelle Gladys
Edit à la main de la DB pour supprimer toutes les feature_state (pour réduire de 5Go à 40Mo la DB) et les t_variable de Gladys Plus et Telegram (pour éviter de faire tourner en double deux instances)
Accès à Z2M pour déclencher un backup
Les logs confirment le backup

2022-11-11T14:32:15+0100 <info> saveZ2mBackup.js:12 (Zigbee2mqttManager.saveZ2mBackup) Zigbee2mqtt: storing backup...
2022-11-11T14:32:16+0100 <info> saveZ2mBackup.js:24 (Zigbee2mqttManager.saveZ2mBackup) Zigbee2mqtt: backup stored

Déplacement de la DB sur mon Raspberry Pi (sur lequel j’ai remis mon adaptateur USB Zigbee), suppression de tous les anciens fichiers dans /var/lib/gladysassistant liés à mon ancienne installation.
Lancement de Gladys avec cette DB sur mon Raspberry Pi
Vérification que ça lance bien les conteneurs Z2M, MQTT et que mes équipement se rattachent au réseau

> gladys-server@ start:prod /src/server
> cross-env NODE_ENV=production node index.js

2022-11-11T14:56:21+0100 <info> job.purge.js:17 (Job.purge) Deleting all background jobs created before = Fri Nov 04 2022 14:56:21 GMT+0100 (Central European Standard Time)
Initialising OpenZWave 1.6.0 binary addon for Node.JS.
        OpenZWave Security API is ENABLED
        ZWave device db    : /usr/local/etc/openzwave
        User settings path : /src/server/services/zwave/node_modules/openzwave-shared/build/Release/../../
        Option Overrides : --Logging false --ConsoleOutput false --SaveConfiguration true
2022-11-11T14:56:24+0100 <info> service.startAll.js:30 () Service alexa was manually stopped, so it is ignored at startup
2022-11-11T14:56:24+0100 <info> service.startAll.js:30 () Service rtsp-camera was manually stopped, so it is ignored at startup
2022-11-11T14:56:24+0100 <info> service.startAll.js:30 () Service xiaomi was manually stopped, so it is ignored at startup
2022-11-11T14:56:24+0100 <info> service.startAll.js:30 () Service zwave was manually stopped, so it is ignored at startup
2022-11-11T14:56:24+0100 <info> service.startAll.js:30 () Service tasmota was manually stopped, so it is ignored at startup
2022-11-11T14:56:24+0100 <info> service.startAll.js:30 () Service bluetooth was manually stopped, so it is ignored at startup
2022-11-11T14:56:24+0100 <info> service.startAll.js:30 () Service ewelink was manually stopped, so it is ignored at startup
2022-11-11T14:56:24+0100 <info> service.startAll.js:30 () Service tp-link was manually stopped, so it is ignored at startup
2022-11-11T14:56:24+0100 <info> service.startAll.js:30 () Service google-actions was manually stopped, so it is ignored at startup
2022-11-11T14:56:24+0100 <info> service.startAll.js:30 () Service homekit was manually stopped, so it is ignored at startup
2022-11-11T14:56:24+0100 <info> service.startAll.js:30 () Service broadlink was manually stopped, so it is ignored at startup
2022-11-11T14:56:24+0100 <info> service.startAll.js:30 () Service philips-hue was manually stopped, so it is ignored at startup
2022-11-11T14:56:24+0100 <info> service.startAll.js:30 () Service example was manually stopped, so it is ignored at startup
2022-11-11T14:56:24+0100 <info> index.js:16 (Object.start) Starting MQTT service
2022-11-11T14:56:24+0100 <info> updateContainer.js:13 (MqttHandler.updateContainer) MQTT: checking for required changes...
2022-11-11T14:56:24+0100 <warn> connect.js:50 (MqttClient.<anonymous>) Error while connecting to MQTT - Error: connect ECONNREFUSED 127.0.0.1:1883
2022-11-11T14:56:24+0100 <info> index.js:20 (Object.start) Starting Open Weather service
2022-11-11T14:56:24+0100 <info> init.js:39 (Zigbee2mqttManager.init) Zigbee2mqtt USB dongle attached to /dev/ttyUSB0
2022-11-11T14:56:24+0100 <info> installMqttContainer.js:28 (Zigbee2mqttManager.installMqttContainer) MQTT broker is being installed as Docker container...
2022-11-11T14:56:24+0100 <info> installMqttContainer.js:29 (Zigbee2mqttManager.installMqttContainer) Pulling eclipse-mosquitto:2 image...
2022-11-11T14:56:30+0100 <info> installMqttContainer.js:33 (Zigbee2mqttManager.installMqttContainer) Preparing broker environment...
2022-11-11T14:56:30+0100 <info> installMqttContainer.js:40 (Zigbee2mqttManager.installMqttContainer) Creating container with data in "/var/lib/gladysassistant" on host...
2022-11-11T14:56:44+0100 <info> installMqttContainer.js:54 (Zigbee2mqttManager.installMqttContainer) MQTT broker is restarting...
2022-11-11T14:57:17+0100 <info> installMqttContainer.js:61 (Zigbee2mqttManager.installMqttContainer) Creating user/pass...
2022-11-11T14:57:17+0100 <info> installMqttContainer.js:70 (Zigbee2mqttManager.installMqttContainer) MQTT broker is restarting...
2022-11-11T14:57:25+0100 <info> installMqttContainer.js:74 (Zigbee2mqttManager.installMqttContainer) MQTT broker container successfully started and configured
2022-11-11T14:57:25+0100 <info> restoreZ2mBackup.js:26 (Zigbee2mqttManager.restoreZ2mBackup) Zigbee2mqtt: loading z2m backup...
2022-11-11T14:57:25+0100 <info> restoreZ2mBackup.js:29 (Zigbee2mqttManager.restoreZ2mBackup) Restoring zigbee2mqtt configuration...
2022-11-11T14:57:25+0100 <info> installZ2mContainer.js:34 (Zigbee2mqttManager.installZ2mContainer) Zigbee2mqtt is being installed as Docker container...
2022-11-11T14:57:25+0100 <info> installZ2mContainer.js:35 (Zigbee2mqttManager.installZ2mContainer) Pulling koenkk/zigbee2mqtt:latest image...
2022-11-11T14:57:26+0100 <info> installZ2mContainer.js:38 (Zigbee2mqttManager.installZ2mContainer) Configuration of Device /dev/ttyUSB0
2022-11-11T14:57:26+0100 <info> installZ2mContainer.js:43 (Zigbee2mqttManager.installZ2mContainer) Creation of container...
2022-11-11T14:57:27+0100 <info> installZ2mContainer.js:46 (Zigbee2mqttManager.installZ2mContainer) Zigbee2mqtt successfully installed and configured as Docker container
2022-11-11T14:57:27+0100 <info> installZ2mContainer.js:59 (Zigbee2mqttManager.installZ2mContainer) Preparing Zigbee2mqtt environment...
2022-11-11T14:57:27+0100 <info> installZ2mContainer.js:72 (Zigbee2mqttManager.installZ2mContainer) Zigbee2mqtt container is starting...
2022-11-11T14:57:33+0100 <info> installZ2mContainer.js:78 (Zigbee2mqttManager.installZ2mContainer) Zigbee2mqtt container successfully started
2022-11-11T14:57:33+0100 <info> connect.js:14 (Zigbee2mqttManager.connect) Connecting Gladys to mqtt://localhost:1884 MQTT broker...
2022-11-11T14:57:33+0100 <info> connect.js:24 (MqttClient.<anonymous>) Connected to MQTT container mqtt://localhost:1884
2022-11-11T14:57:33+0100 <info> subscribe.js:12 (Zigbee2mqttManager.subscribe) Subscribing to MQTT topic zigbee2mqtt/#
2022-11-11T14:57:33+0100 <info> index.js:13 (Object.start) Starting usb service
2022-11-11T14:57:33+0100 <info> index.js:19 (Object.start) Starting Telegram service
2022-11-11T14:57:33+0100 <info> service.start.js:40 (Service.start) Service telegram is not configured, so it was not started.
2022-11-11T14:57:33+0100 <info> index.js:88 (Object.start) Starting CalDAV service
2022-11-11T14:57:33+0100 <info> scene.dailyUpdate.js:58 () Sunrise today is at 7:55 today, in your timezone = Europe/Paris
2022-11-11T14:57:33+0100 <info> scene.dailyUpdate.js:59 () Sunset today is at 17:17 today, in your timezone = Europe/Paris
2022-11-11T14:57:33+0100 <info> scene.dailyUpdate.js:67 () Sunrise is scheduled, 7 hours ago.
2022-11-11T14:57:33+0100 <info> scene.dailyUpdate.js:81 () Sunset is scheduled, in 2 hours.
2022-11-11T14:57:34+0100 <info> device.calculateAggregate.js:38 (DeviceManager.calculateAggregate) Calculating aggregates device feature state for interval hourly
2022-11-11T14:57:34+0100 <info> index.js:63 (Server.<anonymous>) Server listening on port 8088
2022-11-11T14:57:45+0100 <warn> handleMqttMessage.js:106 () Zigbee2mqtt device CapteurMouvementEntree, feature illuminance_lux not configured in Gladys.
2022-11-11T14:57:45+0100 <warn> handleMqttMessage.js:106 () Zigbee2mqtt device capteurQualitéDeLair, feature formaldehyd not configured in Gladys.
2022-11-11T14:57:45+0100 <info> device.calculateAggregate.js:105 (ChildProcess.<anonymous>) device.calculateAggregate: Finishing processing for interval hourly
2022-11-11T14:57:46+0100 <info> device.calculateAggregate.js:38 (DeviceManager.calculateAggregate) Calculating aggregates device feature state for interval daily
2022-11-11T14:57:47+0100 <warn> handleMqttMessage.js:106 () Zigbee2mqtt device capteurQualitéDeLair, feature formaldehyd not configured in Gladys.
2022-11-11T14:57:50+0100 <warn> handleMqttMessage.js:106 () Zigbee2mqtt device capteurQualitéDeLair, feature formaldehyd not configured in Gladys.
2022-11-11T14:57:54+0100 <warn> handleMqttMessage.js:106 () Zigbee2mqtt device capteurQualitéDeLair, feature formaldehyd not configured in Gladys.
2022-11-11T14:57:55+0100 <warn> device.calculateAggregate.js:95 (Socket.<anonymous>) device.calculateAggregate stderr: Error
    at Database.<anonymous> (/src/server/node_modules/sequelize/lib/dialects/sqlite/query.js:179:27)
    at /src/server/node_modules/sequelize/lib/dialects/sqlite/query.js:177:50
    at new Promise (<anonymous>)
    at Query.run (/src/server/node_modules/sequelize/lib/dialects/sqlite/query.js:177:12)
    at /src/server/node_modules/sequelize/lib/sequelize.js:313:28
    at async SQLiteQueryInterface.bulkUpdate (/src/server/node_modules/sequelize/lib/dialects/abstract/query-interface.js:355:12)
    at async Function.update (/src/server/node_modules/sequelize/lib/model.js:1952:28)
    at async /src/server/lib/device/device.calculcateAggregateChildProcess.js:153:5 {
  name: 'SequelizeTimeoutError',
  parent: [Error: SQLITE_BUSY: database is locked] {
    errno: 5,
    code: 'SQLITE_BUSY',
    sql: 'UPDATE `t_device_feature` SET `last_daily_aggregate`=$1,`updated_at`=$2 WHERE `id` = $3'
  },
  original: [Error: SQLITE_BUSY: database is locked] {
    errno: 5,
    code: 'SQLITE_BUSY',
    sql: 'UPDATE `t_device_feature` SET `last_daily_aggregate`=$1,`updated_at`=$2 WHERE `id` = $3'
  },
  sql: 'UPDATE `t_device_feature` SET `last_daily_aggregate`=$1,`updated_at`=$2 WHERE `id` = $3',
  parameters: {}
}

2022-11-11T14:57:55+0100 <warn> device.calculateAggregate.js:101 (ChildProcess.<anonymous>) device.calculateAggregate: Exiting child process with code 1
2022-11-11T14:57:57+0100 <warn> handleMqttMessage.js:106 () Zigbee2mqtt device capteurQualitéDeLair, feature formaldehyd not configured in Gladys.
2022-11-11T14:58:00+0100 <info> scene.checkCalendarTriggers.js:24 (SceneManager.checkCalendarTriggers) Checking calendar triggers at Fri, 11 Nov 2022 13:58:00 GMT
2022-11-11T14:58:01+0100 <warn> handleMqttMessage.js:106 () Zigbee2mqtt device capteurQualitéDeLair, feature formaldehyd not configured in Gladys.
2022-11-11T14:58:04+0100 <warn> handleMqttMessage.js:106 () Zigbee2mqtt device capteurQualitéDeLair, feature formaldehyd not configured in Gladys.
2022-11-11T14:58:08+0100 <error> device.onHourlyDeviceAggregateEvent.js:27 (DeviceManager.onHourlyDeviceAggregateEvent) Error: Error
    at Database.<anonymous> (/src/server/node_modules/sequelize/lib/dialects/sqlite/query.js:179:27)
    at /src/server/node_modules/sequelize/lib/dialects/sqlite/query.js:177:50
    at new Promise (<anonymous>)
    at Query.run (/src/server/node_modules/sequelize/lib/dialects/sqlite/query.js:177:12)
    at /src/server/node_modules/sequelize/lib/sequelize.js:313:28
    at async SQLiteQueryInterface.bulkUpdate (/src/server/node_modules/sequelize/lib/dialects/abstract/query-interface.js:355:12)
    at async Function.update (/src/server/node_modules/sequelize/lib/model.js:1952:28)
    at async /src/server/lib/device/device.calculcateAggregateChildProcess.js:153:5 {
  name: 'SequelizeTimeoutError',
  parent: [Error: SQLITE_BUSY: database is locked] {
    errno: 5,
    code: 'SQLITE_BUSY',
    sql: 'UPDATE `t_device_feature` SET `last_daily_aggregate`=$1,`updated_at`=$2 WHERE `id` = $3'
  },
  original: [Error: SQLITE_BUSY: database is locked] {
    errno: 5,
    code: 'SQLITE_BUSY',
    sql: 'UPDATE `t_device_feature` SET `last_daily_aggregate`=$1,`updated_at`=$2 WHERE `id` = $3'
  },
  sql: 'UPDATE `t_device_feature` SET `last_daily_aggregate`=$1,`updated_at`=$2 WHERE `id` = $3',
  parameters: {}
}

    at ChildProcess.<anonymous> (/src/server/lib/device/device.calculateAggregate.js:102:23)
    at ChildProcess.emit (events.js:400:28)
    at maybeClose (internal/child_process.js:1088:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:296:5)
2022-11-11T14:58:08+0100 <info> device.calculateAggregate.js:38 (DeviceManager.calculateAggregate) Calculating aggregates device feature state for interval monthly
2022-11-11T14:58:08+0100 <warn> handleMqttMessage.js:106 () Zigbee2mqtt device capteurQualitéDeLair, feature formaldehyd not configured in Gladys.
2022-11-11T14:58:11+0100 <warn> handleMqttMessage.js:106 () Zigbee2mqtt device capteurQualitéDeLair, feature formaldehyd not configured in Gladys.
2022-11-11T14:58:15+0100 <warn> handleMqttMessage.js:106 () Zigbee2mqtt device capteurQualitéDeLair, feature formaldehyd not configured in Gladys.
2022-11-11T14:58:18+0100 <warn> handleMqttMessage.js:106 () Zigbee2mqtt device capteurQualitéDeLair, feature formaldehyd not configured in Gladys.
2022-11-11T14:58:21+0100 <warn> handleMqttMessage.js:106 () Zigbee2mqtt device CapteurMouvementCuisine, feature illuminance_lux not configured in Gladys.
2022-11-11T14:58:21+0100 <info> device.calculateAggregate.js:105 (ChildProcess.<anonymous>) device.calculateAggregate: Finishing processing for interval monthly

Et …

ça fonctionne super bien !!

Mon interface Gladys me permet de contrôler mes interrupteurs muraux, ma lumière dans le salon s’allume ! ça confirme que le réseau Zigbee s’est à nouveau remis en place avec les mêmes paramètres.

Très très beau boulot !!

Comme tu peux le voir dans les logs, il y a une erreur de DB Lock, qui a été générée lors de mes deux tests de restauration. Mais c’est passager, possiblement sans lien, et surtout Gladys fonctionne bien.

AlexTrovato · November 11, 2022, 2:21pm

Glad to hear and see that!!!

Regarding the DB lock error, I think that following my changes to the shared scheduler, we should no longer trigger data aggregation at startup, but let the scheduler run at its scheduled time.
@pierre-gilles what do you think?

pierre-gilles · November 12, 2022, 4:19am

Awesome!!

Why not remove aggregation at startup

guim31 · November 22, 2022, 6:41pm

Since I’m going to receive my NUC this week, I was wondering what the status of this work is?
Actually I didn’t quite understand whether the work you did will allow a migration « in a few clicks »? Or is the idea that z2m is backed up on its own and that a restore via Gladys Plus will restore it as well?

GBoulvin · November 22, 2022, 7:24pm

It’s still being tested…
I hope to have a bit of time tomorrow or Thursday to look into it.
If I’m not mistaken, everything is supposed to end up in the Gladys DB so it can also be sent via gladys+.

guim31 · November 23, 2022, 4:28am

Okay, thanks for the information!

I should point out that even though I’m impatient (new toy) I wouldn’t allow myself to complain about how long this work takes!

GBoulvin · November 23, 2022, 10:36pm

Tested and approved :

Fresh installation
Basic configuration and addition of a ZigBee switch
Waiting for the automatic backup at 23:00
Stopping the containers
Removal of the subfolders present in /Var/lib/gladysassistant and of all containers
Restarting the Gladys container
And everything returns to how it was, ZigBee is functional!
The logs :
At 23:00 :

2022-11-23T23:00:00+0100 backup.js:39 (Zigbee2mqttManager.backup) Zigbee2MQTT request for backup
2022-11-23T23:00:00+0100 handleMqttMessage.js:84 (Zigbee2mqttManager.handleMqttMessage) Zigbee2mqtt topic zigbee2mqtt/bridge/request/backup empty message.
2022-11-23T23:00:10+0100 device.calculateAggregate.js:105 (ChildProcess.) device.calculateAggregate: Finishing processing for interval hourly
2022-11-23T23:00:10+0100 device.calculateAggregate.js:38 (DeviceManager.calculateAggregate) Calculating aggregates device feature state for interval daily
2022-11-23T23:00:12+0100 saveZ2mBackup.js:12 (Zigbee2mqttManager.saveZ2mBackup) Zigbee2mqtt: storing backup…
2022-11-23T23:00:13+0100 saveZ2mBackup.js:24 (Zigbee2mqttManager.saveZ2mBackup) Zigbee2mqtt: backup stored

After deletion and restart

gladys-server@ start:prod /src/server
cross-env NODE_ENV=production node index.js

2022-11-23T23:17:31+0100 job.purge.js:17 (Job.purge) Deleting all background jobs created before = Wed Nov 16 2022 23:17:31 GMT+0100 (Central European Standard Time)
Initialising OpenZWave 1.6.0 binary addon for Node.JS.
OpenZWave Security API is ENABLED
ZWave device db : /usr/local/etc/openzwave
User settings path : /src/server/services/zwave/node_modules/openzwave-shared/build/Release/../../
Option Overrides : --Logging false --ConsoleOutput false --SaveConfiguration true
2022-11-23T23:17:43+0100 index.js:14 (Object.start) starting Alexa service
2022-11-23T23:17:43+0100 index.js:21 (Object.start) Starting example service
2022-11-23T23:17:44+0100 index.js:20 (Object.start) Starting Open Weather service
2022-11-23T23:17:44+0100 service.start.js:40 (Service.start) Service openweather is not configured, so it was not started.
2022-11-23T23:17:44+0100 index.js:88 (Object.start) Starting CalDAV service
2022-11-23T23:17:44+0100 index.js:16 (Object.start) starting Broadlink service
2022-11-23T23:17:44+0100 index.js:18 (Object.start) Starting HomeKit service
2022-11-23T23:17:45+0100 index.js:14 (Object.start) starting GoogleActions service
2022-11-23T23:17:46+0100 init.js:39 (Zigbee2mqttManager.init) Zigbee2mqtt USB dongle attached to /dev/ttyUSB0
2022-11-23T23:17:46+0100 installMqttContainer.js:28 (Zigbee2mqttManager.installMqttContainer) MQTT broker is being installed as Docker container…
2022-11-23T23:17:46+0100 installMqttContainer.js:29 (Zigbee2mqttManager.installMqttContainer) Pulling eclipse-mosquitto:2 image…
2022-11-23T23:17:47+0100 installMqttContainer.js:33 (Zigbee2mqttManager.installMqttContainer) Preparing broker environment…
2022-11-23T23:17:47+0100 installMqttContainer.js:40 (Zigbee2mqttManager.installMqttContainer) Creating container with data in « /var/lib/gladysassistant » on host…
2022-11-23T23:17:49+0100 installMqttContainer.js:54 (Zigbee2mqttManager.installMqttContainer) MQTT broker is restarting…
2022-11-23T23:17:55+0100 installMqttContainer.js:61 (Zigbee2mqttManager.installMqttContainer) Creating user/pass…
2022-11-23T23:17:56+0100 installMqttContainer.js:70 (Zigbee2mqttManager.installMqttContainer) MQTT broker is restarting…
2022-11-23T23:18:03+0100 installMqttContainer.js:74 (Zigbee2mqttManager.installMqttContainer) MQTT broker container successfully started and configured
2022-11-23T23:18:03+0100 restoreZ2mBackup.js:26 (Zigbee2mqttManager.restoreZ2mBackup) Zigbee2mqtt: loading z2m backup…
2022-11-23T23:18:03+0100 restoreZ2mBackup.js:29 (Zigbee2mqttManager.restoreZ2mBackup) Restoring zigbee2mqtt configuration…
2022-11-23T23:18:03+0100 installZ2mContainer.js:34 (Zigbee2mqttManager.installZ2mContainer) Zigbee2mqtt is being installed as Docker container…
2022-11-23T23:18:03+0100 installZ2mContainer.js:35 (Zigbee2mqttManager.installZ2mContainer) Pulling koenkk/zigbee2mqtt:latest image…
2022-11-23T23:18:05+0100 installZ2mContainer.js:38 (Zigbee2mqttManager.installZ2mContainer) Configuration of Device /dev/ttyUSB0
2022-11-23T23:18:05+0100 installZ2mContainer.js:43 (Zigbee2mqttManager.installZ2mContainer) Creation of container…
2022-11-23T23:18:08+0100 installZ2mContainer.js:46 (Zigbee2mqttManager.installZ2mContainer) Zigbee2mqtt successfully installed and configured as Docker container
2022-11-23T23:18:08+0100 installZ2mContainer.js:59 (Zigbee2mqttManager.installZ2mContainer) Preparing Zigbee2mqtt environment…
2022-11-23T23:18:08+0100 installZ2mContainer.js:72 (Zigbee2mqttManager.installZ2mContainer) Zigbee2mqtt container is starting…
2022-11-23T23:18:14+0100 installZ2mContainer.js:78 (Zigbee2mqttManager.installZ2mContainer) Zigbee2mqtt container successfully started
2022-11-23T23:18:14+0100 connect.js:14 (Zigbee2mqttManager.connect) Connecting Gladys to mqtt://localhost:1884 MQTT broker…
2022-11-23T23:18:15+0100 connect.js:24 (MqttClient.) Connected to MQTT container mqtt://localhost:1884
2022-11-23T23:18:15+0100 subscribe.js:12 (Zigbee2mqttManager.subscribe) Subscribing to MQTT topic zigbee2mqtt/#
2022-11-23T23:18:15+0100 index.js:18 (Object.start) Starting TP-Link service
2022-11-23T23:18:15+0100 index.js:17 (Object.start) Starting eWeLink service
2022-11-23T23:18:15+0100 service.start.js:40 (Service.start) Service ewelink is not configured, so it was not started.
2022-11-23T23:18:16+0100 index.js:15 (Object.start) Starting Bluetooth service
2022-11-23T23:18:18+0100 index.js:20 (process.) uncaughtException catched: uncaughtException
2022-11-23T23:18:18+0100 index.js:21 (process.) Error: ENODEV, No such device
at Hci.init (/src/server/services/bluetooth/node_modules/@abandonware/noble/lib/hci-socket/hci.js:100:35)
at NobleBindings.init (/src/server/services/bluetooth/node_modules/@abandonware/noble/lib/hci-socket/bindings.js:78:13)
at /src/server/services/bluetooth/node_modules/@abandonware/noble/lib/noble.js:60:24
at processTicksAndRejections (internal/process/task_queues.js:77:11) {
errno: 19,
code: ‹ ENODEV ›,
syscall: ‹ bind ›
}
2022-11-23T23:18:18+0100 index.js:15 (Object.start) Starting Tasmota service
2022-11-23T23:18:18+0100 index.js:16 (Object.start) Starting zwave service
2022-11-23T23:18:18+0100 service.start.js:40 (Service.start) Service zwave is not configured, so it was not started.
2022-11-23T23:18:18+0100 index.js:14 (Object.start) Starting Xiaomi service
2022-11-23T23:18:18+0100 index.js:13 (Object.start) Starting usb service
2022-11-23T23:18:18+0100 index.js:19 (Object.start) Starting Telegram service
2022-11-23T23:18:18+0100 service.start.js:40 (Service.start) Service telegram is not configured, so it was not started.
2022-11-23T23:18:18+0100 index.js:16 (Object.start) Starting RTSP service
2022-11-23T23:18:18+0100 index.js:18 (Object.start) Starting Philips Hue service
2022-11-23T23:18:19+0100 index.js:16 (Object.start) Starting MQTT service
2022-11-23T23:18:19+0100 updateContainer.js:13 (MqttHandler.updateContainer) MQTT: checking for required changes…
2022-11-23T23:18:19+0100 service.start.js:40 (Service.start) Service mqtt is not configured, so it was not started.
2022-11-23T23:18:19+0100 scene.dailyUpdate.js:58 () Sunrise today is at 8:10 today, in your timezone = Europe/Paris
2022-11-23T23:18:19+0100 scene.dailyUpdate.js:59 () Sunset today is at 16:50 today, in your timezone = Europe/Paris

Tomorrow, I’ll test with my production DB.
Good job !

pierre-gilles · November 24, 2022, 2:37am

Thanks for testing @GBoulvin! Super clean!

I need to review the code now — I’ll keep you posted as soon as I have time.

VonOx · November 24, 2022, 11:45am

Alex was able to develop on real hardware, so I’m not too worried about the functionality.

lmilcent · November 24, 2022, 6:32pm

And I also confirmed it with my production database on a Raspberry Pi!

pierre-gilles · November 25, 2022, 7:23am

I did an initial review of the PR « common scheduler », which is a prerequisite before merging the Zigbee2mqtt backup PR.

Since the other PR incorporates the changes

pierre-gilles · November 25, 2022, 7:54am

I’ve merged the first PR of the series of 3 PRs needed for this feature:

github.com/GladysAssistant/Gladys

Z2m clean code

master ← atrovato:z2m-clean-code

ouvert 05:39AM - 01 Nov 22 UTC

atrovato

+744 -591

### Pull Request check-list To ensure your Pull Request can be accepted as fa…st as possible, make sure to review and check all of these items: - [x] If your changes affects code, did your write the tests? - [x] Are tests passing? (`npm test` on both front/server) - [x] Is the linter passing? (`npm run eslint` on both front/server) - [x] Did you run prettier? (`npm run prettier` on both front/server) - ~~[ ] If you are adding a new features/services, did you run integration comparator? (`npm run compare-translations` on front)~~ - [x] Did you test this pull request in real life? With real devices? If this development is a big feature or a new service, we recommend that you provide a Docker image to [the community](https://community.gladysassistant.com/) for testing before merging. - ~~[ ] If your changes modify the API (REST or Node.js), did you modify the API documentation? (Documentation is based on comments in code)~~ - ~~[ ] If you are adding a new features/services which needs explanation, did you modify the user documentation? See [the GitHub repo](https://github.com/GladysAssistant/v4-website) and the [website](https://gladysassistant.com).~~ - ~~[ ] Did you add fake requests data for the demo mode (`front/src/config/demo.js`) so that the demo website is working without a backend? (if needed) See [https://demo.gladysassistant.com](https://demo.gladysassistant.com).~~ NOTE: these things are not required to open a PR and can be done afterwards / while the PR is open. ### Description of change This PR prepares the "z2m backup" feature. The z2m code is a bit disorganized, so to add features in it, it looks difficult and risky. So I just ran a cleaning and organization phase. It cleans the current z2m service code to: - use "debug" logger instead of "log" - load and store variables in Gladys database in a unique method (instead of everywhere in methods) - clean tests when proxyquire is not required - implements afterEach when needed on tests - ... I tested it in real life, in both upgrading and new install modes. It looks good.

VonOx · November 25, 2022, 8:54am

@AlexTrovato is a machine

guim31 · December 27, 2022, 9:21am

Hi everyone,

I’m checking in because, since I don’t understand developers’ language (merging a PR etc…) I have no idea what the status of this backup feature is?
Do tests need to be done? Are testers needed?

spenceur · December 28, 2022, 5:51am

Basically, the first PR out of 3 is on the production branch.
Il faudra attendre les deux autres pour que la fonctionnalité soit 100% fonctionnelle.
Mais pour la premières tout est passé test compris.

guim31 · December 28, 2022, 2:52pm

Ok thanks @spenceur for this status update!

Topic		Replies	Views
Gladys Assistant 4.2 est disponible avec la compatibilité Zigbee2mqtt ! 🚀 Actualités	146	5617	May 17, 2021
Zigbee2mqtt : Image docker de test basée Gladys v4 Développement	409	12921	September 5, 2021
Remettre en place le fichier gladys-production.db Configuration	11	655	May 2, 2022
Nouvelle version 4.7 : Affichage graphique et mise à jour majeure de l'intégration Zigbee2mqtt! Actualités	47	3928	April 8, 2022
Utilisation du disque par les logs Mosquitto Développement zigbee2mqtt	29	1180	February 6, 2023

Backup of the zigbee2mqtt database

Related topics