📈 Transition to Node.js LTS 16/18 + end of Open-Zwave

Hi @contributors!

I merged the PR that removes the Open-Zwave integration (Remove Open-Zwave integration by Pierre-Gilles · Pull Request #1617 · GladysAssistant/Gladys · GitHub), which unblocks a bunch of tasks for us :slight_smile:

What I have in mind:

  1. Upgrade to Node.js 16/18
  2. Switch to the WebCrypto API for Gladys Plus instead of the polyfill we were using until now
  3. Update server dependencies
  4. Update front-end dependencies
  5. Update GitHub Actions
  6. Update Cypress

What I’ve done so far:

1., 2., 3. (partially) and 5. in this PR:

For now I’m switching to Node.js 18 on the server, and Node.js 16 on the front-end, because preact-cli runs on Webpack 4 which is not compatible with Node.js 18 yet.

@Pti_Nico I based this on your work in your PR (ci: update Cypress to 10 by NickDub · Pull Request #1630 · GladysAssistant/Gladys · GitHub), could you maybe open a separate PR for Cypress? (Or wait until I merge my Node 18 PR)

Before/After

I didn’t think it would be this impressive, but basically here’s the execution speed of PR tests:

Before (example on the last PR)

Now (on my PR):

7 Likes

It just goes to show that keeping software and technologies up to date is important and actually useful :blush:

I have to admit that’s a pretty crazy performance gain!

There are several reasons for this improvement; it’s not just upgrading to Node.js 16/18 that’s the key:

  • Open-Zwave was a library based on C++ bindings that had to be built at each npm install + docker build for every architecture. Removing this lib contributed a lot.
  • Before Node.js 16, the WebCrypto API wasn’t available in Node.js, so I used a polyfill that had to be compiled at each npm install + each docker build for every architecture.

In the gateway code, the change looks like this:

This removes node-webcrypto-ossl, which was a nightmare to compile, and we go through Node.js’s native API.

Potentially, this improves the performance and stability of the Gladys Plus part because we use a rock-solid, official native API.

  • We’re moving to a more recent version of sqlite3 which has proper bindings with the Node-API, and which is normally pre-compiled for each architecture; likewise, we save a build per npm install + docker build per architecture.

In short, the ecosystem has evolved a lot and not using the latest version of Node prevented us from adopting many new optimizations.

Another advantage of all this is that we’ll have a much easier developer onboarding, which will remove a barrier to developing Gladys services.

We’ll be able to update the tutorials, and it will be much simpler :slight_smile:

3 Likes

I have a first Docker image running on Node.js 18:

gladysassistant/gladys:upgrade-node-18

I’m open to feedback :slight_smile:

Edit: dev tag now:

1 Like

I admit that, for the dev environment, compiling the Z-Wave module was a pain!

1 Like

[quote=« pierre-gilles, post:1, topic:7708 »]
@Pti_Nico I based this on your work on your PR ( ci: update CI deprecated actions by NickDub · Pull Request #1630 · GladysAssistant/Gladys · GitHub ), could you maybe create a separate PR for Cypress? (Maybe wait until I merge my

1 Like

FYI, I incorporated the various feedback on the PR and merged the PR into master so that all ongoing PRs can adapt and benefit from the build speed improvements! :partying_face:

I updated the Linux/MacOS developer documentation, which is now much clearer in my opinion — open-zwave is done :slight_smile:

FYI, in the end I moved to Node.js 18 on the back and the front, so no Node 16.

For the Windows documentation, could someone give it a pass and check that everything works well with Node.js 18 LTS on WSL and update the documentation? Thanks!

The Docker image on the :dev tag is now based on Node.js 18 LTS:

gladysassistant/gladys:dev

On Apple M1 Pro, it’s really impressive now, the 1785 backend tests run in 1 minute :sweat_smile: I’m saving a ton of development time!

And there’s still room for improvement, there are 4-5 tests on the MQTT and Zigbee2mqtt services that take 10 seconds each, so almost half of the total test time is spent on these integrations.

I created GitHub issues for these tests:

2 Likes

@pierre-gilles I’ve reviewed my PR :wink:

1 Like

It’s done

1 Like

Great! It’s good for me, merged :slight_smile:

Efficient :smiley: It’s merged, the new Windows tutorial is live on the site:

1 Like