External integrations in Gladys Assistant

Status: draft, open for discussion
Discussion: this forum topic


Hi everyone,

I’m proposing a new feature that will change the way integrations are developed in Gladys.

This is a long-considered project, and I would be delighted to have your feedback, your ideas, and your comments to continue improving it :slight_smile:

Why now

Since the beginning of the project, all Gladys integrations have lived in the core. Anyone can develop one or improve it via a pull request, but every line goes through my review before being merged. This choice has a huge advantage: when you install Gladys, everything is already there. No store, no dependencies to manage, no broken plugins after an update. This is one of the reasons why Gladys is easier to get started with than other solutions.

But this model has a ceiling, and I’m hitting it. There are thousands of brands, protocols, services, and even the slightest modification of an integration, even a simple translation change, must go through me: review, merge, release. It doesn’t scale, and it makes me the bottleneck of the project.

One might think that Matter will solve this problem. Matter is coming, and in my opinion, it will become the reference protocol that will control all connected devices in the future: one standard, all devices natively compatible, no need for an integration per brand. Except that we don’t know when this future will be a reality, and the project cannot afford to wait indefinitely for the installed base to switch.

And above all, Matter only covers the control of connected devices. There is still a whole range of integrations that have nothing to do with devices: communication (Telegram, etc.), weather (OpenWeather, Météo France), calendars (CalDAV, Google Calendar)… None of this will ever go through Matter. If we want Gladys to become a project with the same ambition as Home Assistant, we must be able to allow the installation of external integrations.

This RFC therefore proposes to open Gladys to external integrations: integrations developed and published by anyone, without prior validation from me, installable in one click from the interface.

The challenge is to do this without sacrificing what makes Gladys. Concretely, four non-negotiable requirements have guided this proposal:

  1. An integration that crashes must never crash Gladys.
  2. No incomprehensible states: if an integration no longer responds, the user must see it and be able to act.
  3. The interfaces must remain clean and consistent with each other.
  4. Zero technical manipulation for the user. No terminal, no YAML, no manual restart.

What this RFC does not propose

  • We are not replacing native integrations. Integrations of universal protocols (Matter, Zigbee, Z-Wave…) will always remain in the core, pre-installed, maintained as today. The new system is added alongside, and targets all integrations of protocols or services that are not universal: from now on, anyone will be able to create a separate integration.
  • We are not aiming for compatibility with Home Assistant / HACS integrations. They are deeply coupled to HA’s architecture; running them in Gladys is unrealistic. However, I am inspired by their distribution model (Git repository + manifest + store).
  • We do not allow integrations to inject code into the interface. This is a deliberate choice, detailed below.

Proposed Architecture

A Docker container per integration

Gladys already runs exclusively in Docker, with the Docker socket mounted. We rely on this: each external integration runs in its own container, created and supervised by the Gladys core.

This container is locked to the maximum:

  • no privileged mode, no access to the Docker socket
  • no access to the database or Gladys files
  • read-only file system, except for a small data directory that is its own
  • strict limits on memory, CPU, and number of processes
  • network restricted to the hosts that the integration has declared in its manifest.

An integration that crashes, leaks memory, or goes into an infinite loop remains confined in its box. And a malicious or poorly written integration cannot « go modify the database live »: the SQLite file simply does not exist in its file system.

This choice brings an important bonus: integrations are no longer limited to Node.js. A Docker image can contain Python, Go, Rust. Each author embeds their dependencies in their image, and all the build is done upstream, at the time of publication, never at runtime at the user’s end.

In the long term, this choice also brings something interesting: it will be possible to run remote integrations!

All communication goes through the host API

An integration never touches Gladys’ internals. Its only entry point is a host API: the integration communicates with the core via a simple HTTP JSON REST API, in the same spirit as what already exists in Gladys today.

The primitives envisioned for v1:

  • declare devices and their features, in the existing Gladys device/feature model;
  • publish states and receive commands
  • store its configuration and secrets (stored in the DB on the core side)
  • write structured logs
  • request a mediated network discovery: it is the core, which has access to the host network, that performs the mDNS scans and the passthrough of USB devices, then transmits the results. The integration thus never needs the host mode or direct hardware access.

This protocol-level contract is the true foundation of the proposal. The container is « only » the mechanism that makes this contract impossible to bypass. It also allows the internal schema of Gladys to evolve without breaking the ecosystem: as long as the host API is stable, the integrations continue to work.

Supervision: no zombie states

The core includes a supervisor that manages the complete lifecycle of each integration, with a state machine always visible in the interface:

Installed → Starting → Running → Degraded → Faulted → Stopped

The supervisor sends a regular heartbeat to each integration. No response? The integration goes to « Degraded » and is automatically restarted, with an increasing delay between attempts. If it crashes in a loop, we stop insisting: it goes to « Faulted », and the user sees a clear message with the integration logs and possible actions (restart, disable, report to the developer).

All calls between the core and the integration have a timeout. An integration that hangs never blocks Gladys.

The goal: there should be no more unobservable or unrecoverable states. When something goes wrong, we see it, we understand it, we act, from the interface.

Interface: declarative, no arbitrary code

This is probably the most debatable choice of this RFC, so it’s best to clearly assume it.

Integrations do not provide interface components. They describe their needs, and it’s Gladys that renders the interface with its own design system:

  • the configuration is described by a schema (type JSON Schema): Gladys generates the form, the validation, the error messages, in an identical way for all integrations;
  • devices are declared in the existing device/feature model: they are automatically displayed with the same cards and the same controls as native integrations;
  • for more specific needs, a vocabulary of declarative widgets will be defined and progressively enriched, according to the real needs reported by developers.

What it costs: a developer won’t be able to create a fully custom screen. What it guarantees: a consistent interface everywhere, no XSS vulnerabilities from a plugin, no front-end version conflicts, and an identical experience for the user regardless of the integration’s origin. Given the project’s priorities, I think this is the right compromise. But this is exactly the kind of point on which I look forward to your feedback.

Distribution: a store in Gladys

The user discovers and installs integrations from a store built into the interface. Installing = one click. The core downloads the image, verifies it, launches the container, and displays the configuration form if the integration requires credentials. No terminal, no files to edit.

Each integration is published with a manifest: name, version, compatible Gladys versions, requested permissions (network hosts, devices), configuration schema. Permissions are displayed to the user before installation, just like on a mobile store.

All external integrations are „community based‟: I don’t plan to review them one by one, that would fall back into the bottleneck that this RFC is trying to eliminate. A clear warning is displayed during installation to remind users that this is unaudited third-party code. It’s the role of the declared permissions and container isolation to make this openness acceptable.

Developing an integration in the era of AI

One thing that changes everything compared to a few years ago: developing an integration has become very easy thanks to AI. My intention is to provide an official, clean, and documented integration template. Starting from this template, with Claude Code/Cursor, creating an integration for a given service or protocol becomes a matter of a few hours rather than a few days.

This is what makes this proposal truly powerful: instead of waiting for me to develop or review each integration, the community will be able to produce many more, much faster.

The pace of new integrations will no longer be limited by my availability.

Proposed roadmap

  1. Define the host API and SDK, then propose to a few developers to create the first external integrations on it.
  2. Specify the manifest and the declarative interface schema.
  3. Implement the supervisor and the launch of locked containers, as a proof of concept on a single integration.
  4. Build the store and the installation process, with the official integration template.
  5. Open publication to everyone.

My goal is to release this new system as quickly as possible: with AI, it’s possible to move fast.

I plan to start the work this week with Fable 5, while it’s available.

Open questions to the community

  • Does the „declarative UI only‟ compromise seem acceptable to you? What real use cases wouldn’t fit into this?
  • What primitives are missing from the host API for your dream integrations?
  • Should we limit v1 to Node.js integrations (with an official SDK) to simplify, or open all languages from the start?

This proposal is a starting point, not a decision set in stone: your criticisms, objections, and ideas are exactly what I’m looking for to make it better :slight_smile:

Hello @pierre-gilles,

I’m not an expert in home automation, but I’m of course very happy to hear about this possibility of custom integration to meet specific needs!

I assume that if an extension is highly relevant and in high demand, it will eventually be integrated into the Gladys core?

Will there be any performance issues with having a hundred Docker containers going through a single API Host channel?

I’ve been using Gladys for less than two months, but I’m amazed by its responsiveness and the speed of its development!

Thanks again!

It’s indeed a big change but I also think it’s necessary. You alone can’t develop, review, and maintain all the integrations even with AI. Even with super contributors, reviewing contributions surely takes you a lot of time.

Also, even if it’s to be preferred, we must also recognize that the MatterBridge integration can have its limits when the Matter protocol doesn’t support certain features.

I agree with your approach of clearly separating the core and the integrations. Not allowing integrations to access the system is also a very good point. I would also specify that integrations must be free and open source to be accepted.

As for a v1 version, I would add the Python language, which is widely used in home automation and easy to pick up. Moreover, AI agents work very well for generating Python code.

Good question, but don’t worry: the API is not a single queue where requests are processed one by one. It’s an HTTP server, exactly like the one Gladys already exposes today for the frontend.

Since Node is asynchronous, while a request waits for the database, the event loop processes many others in parallel. 100 containers hitting it is a ridiculous load.

The real point of attention isn’t the API; it’s the RAM. A container itself weighs almost nothing (it’s not a VM), but each Node process carries the V8 runtime, so you need to account for 50-80 MB of RAM used per integration depending on what it does. With a dozen integrations, no problem on a mini-PC. With a hundred running simultaneously, that depends on your available RAM :smiley:

In practice, most installations will have between 5 and 20 external integrations, which is totally comfortable on most mini-PCs.

Thanks!!

Anyway, the API will be fully open and documented, so each developer can use it as they wish :wink:

The real issue is more about what examples we provide and in which languages. Personally, I don’t have any experience with Python, so I might have an AI generate an example, but I wouldn’t really be able to review it, validate it, or effectively help users with it.

But that’s the strength of these external integrations: everyone will be free, and I won’t be the bottleneck :smiley:

For the future integration store, I want to go with a fully decentralized architecture, without manual validation, without review, and without needing permission to publish.

In concrete terms, each developer will simply publish their integration in their own GitHub repository by adding a specific topic. This repository will remain the source of truth: everyone keeps control of their code and their updates.

On the other hand, a fully automatic indexer (a public GitHub Action) will regularly scan all repositories with this topic. It will only check objective criteria: presence of a valid manifest, adherence to the schema, accessibility of images, version consistency, etc. If everything is compliant, the integration will be automatically added to a static index.json, which will then be distributed via GitHub Pages and a CDN.

Gladys instances will only download this static index, ensuring high performance, avoiding GitHub API limitations, and making it easy to cache the content. And even if the indexer becomes unavailable, the last published index will continue to work.

The goal is simple: no one needs my approval to publish an integration. The only rules are those automatically checked by the script, whose code will also be public. Thus, everyone will be able to immediately understand why their integration is indexed… or why it is not.

Well, I didn’t understand everything :face_with_head_bandage:, but I think the idea is excellent because it will allow Gladys to make a leap forward in terms of its ability to compete with HA. And this will allow everyone to contribute « easily » to improving Gladys, which will keep its core intact!

It would be great to eventually allow the choice to use another Git hosting platform like GitLab and Codeberg.

Hi @contributors :waving_hand:

I made great progress on this topic today! I spent the day writing the technical specification with Fable 5.

The implementation specification is available here:

My goal is to advance this development quickly, while maintaining a high level of quality.

As always, all your feedback is welcome. :slightly_smiling_face:

3 new repositories created:

Implementation in progress by 3 Fable 5 agents:

I’m getting ahead because Fable 5 is not guaranteed to stay after July 19th.

If you have any feedback, of course everything is still modifiable, I’m really open to feedback!

I think it’s a good thing that integrations can be managed this way.

No language barrier and time saved on your side, as validating each PR takes a lot of time, especially right now with AI.

However, concretely, if we take the example of the Meteo France integration, there is the API part to manage with the integration, but also the widget part and the scene trigger and scene action.
Should we split it into several parts to separate the integration part from the widgets and scene part? There would therefore be a PR for the core with the widgets/scene and the Meteo France integration part for the Gladys store?

And what about the existing integrations? Should they move to the store or will you keep them?

In the long run, I think many integrations can be moved out of the core, but it’s important to clearly state what belongs to the core and what is external. Since external integrations cannot access the hardware, we can already say that the core handles « protocols » like Zigbee, Z-Wave, and Matter because we need to access the USB dongles. The user interface and the supervisor are also part of the core. A list with explanations would be welcome.

To add a new language more easily, I propose two things:

  • Provide a « JSON Schema » to validate the structure of JSON messages
  • A Docker server that allows validating an SDK. The idea would be as follows:
    • I develop an SDK in language X based on the official JS version with the same validation tests
    • Once development is complete, I launch the SDK validation server
    • I run the validation tests by connecting to the server
    • My SDK is validated when 100% of the tests are valid
      This method also allows automating development and validation by an AI agent.

Another point is how much we need to limit the resources of external integrations to avoid having monsters and breaking the Gladys experience. Depending on the languages, this can range from simple to triple in terms of memory and CPU load. If we’re too strict, we close the door to many interpreted languages (Python, Ruby, and even JS with NodeJS), and if it’s too loose, we’ll need a machine with a lot of RAM. So, goodbye to machines like Raspberry Pis, NAS, Freebox, etc. Plus, RAM is now very expensive.
As a C/C++ developer, I’m more on the strict side :slight_smile:

Quick update: Fable 5 did great work yesterday and generated PRs on all relevant repositories, almost exactly following the specification I wrote on Monday. :rocket:

On my side, I will move on to the review and testing phase by the end of the week.

It’s quite impressive, as this is a project that, just a few years ago, would probably have required several months of development. If everything goes as planned, we should be able to release something very quickly.

I’ll keep you posted on the next steps! :slightly_smiling_face:


The idea is really to completely decouple the integrations from the core and have a clear and stable API surface between the two.

For now, the specification and implementation only cover the devices part. I haven’t yet addressed the weather, communications, or other domains, but they will naturally be added following the same model.

To give you an idea, an integration will look like this:

Source: GitHub - GladysAssistant/integration-sdk-js: Gladys Assistant SDK for JS external integration · GitHub

However, if an integration requires a new core capability (for example, a new device type), it will indeed be necessary to:

  1. Make a PR on the core to add this feature.
  2. Once this feature is available, it will be automatically exposed via the API and can be used in external integrations.

It will depend on the integration.

All universal integrations (Matter, Zigbee, etc.) are meant to stay in the core. My philosophy hasn’t changed: the goal is above all to offer the best possible user experience, and these building blocks must be native, without additional installation.

On the other hand, integrations very specific to certain manufacturers or services (for example, Mitsubishi) will gradually migrate to the store.


External integrations will indeed be able to access the hardware, but via APIs provided by the core.

This is actually planned in the specification:

request a mediated network discovery: it’s the core, which has access to the host’s network, that performs the mDNS scans and USB device passthrough, then transmits the results. The integration thus never needs the host mode or direct hardware access.

The goal is to offer all necessary capabilities without compromising security.

And if an integration really needs particular permissions, I am not opposed to a system of explicit permissions, a bit like on iOS or Android, where the user validates the requested accesses.


Yes, it’s already planned in the specification: Gladys/PLAN_INTEGRATIONS_EXTERNES.md at claude/focused-turing-q9vf41 · GladysAssistant/Gladys · GitHub

For now, each integration is limited to:

  • 0.5 vCPU
  • 256 MB of RAM
  • 100 maximum PID (protection against fork bombs)

These values are a starting point and can be adjusted based on the real needs of the integrations and community feedback.

Hi Pierre-Gilles,

A timely topic: I’m currently working on two integrations that are the two extreme cases of your model — Zendure (cloud + token + polling, which you actually mention as an example :grinning_face_with_smiling_eyes:) and a local Frigate/go2rtc video surveillance system (see my post) that orchestrates companion Docker containers, GPU, and video streams. My feedback comes from these two experiences.

I’m up for serving as a test bench when you prototype the host API — Frigate for the « external system supervision » side, Zendure for the « simple cloud » side. I’d prioritize Zendure in this case ^^

To test the limits of each integration you’ve decided on, Frigate would be the right candidate… Because in this case, I’m sure it won’t be enough. For example, 256MB of RAM is the recommended baseline for one camera, and for two, I had to increase it to 384MB.

Thanks @Terdious, this is amazing! We’ll be able to integrate all of this into external integrations!!

I’m currently testing it, and it’s quite mind-blowing. The user experience is really the same as native, but it’s just much simpler to develop, as you only need to develop the backend, and no PR reviews are required :slight_smile: You can go wild :joy:

I’ve converted the native Melcloud integration into an external integration (GitHub - GladysAssistant/gladys-melcloud: Gladys Assistant external integration for Mitsubishi AC · GitHub), of course one-shotted by Fable 5.

Example of a display of a « community » integration vs « native » :

The integration list also includes both native and community integrations, coming from the store.

It’s also possible to launch an integration from Github :

Or from a Docker image for testing :

Then, not many differences with a classic integration :

In short, it really works well!!

I think that if you want to start a Zendure development, you can start now to test and give me feedback on the overall experience.

The external integration template you can give to Claude as a source: GitHub - GladysAssistant/integration-template-js: Gladys Assistant Integration template for a JS external integration · GitHub

The JS SDK: GitHub - GladysAssistant/integration-sdk-js: Gladys Assistant SDK for JS external integration · GitHub

To test in Gladys, you can launch Gladys locally on your machine on this branch: External integrations (RFC phase 1): Docker supervisor, host API, integration WebSocket, decentralized store and generic frontend by Pierre-Gilles · Pull Request #2665 · GladysAssistant/Gladys · GitHub

And in the « Integrations » tab, you will find the « Install from Github » button.

Yes, Frigate is really a special case. The main challenge concerns the container architecture.

Today, an external integration corresponds to a dedicated container. With Frigate, two approaches are possible:

  • launch three distinct containers (Frigate + Mosquitto + the Gladys Frigate external integration);
  • or create a Docker image based on Frigate that directly embeds the Gladys integration code.

Each approach has its advantages and disadvantages. This is a case that is simply not handled in the current architecture, so this point will need to be evolved. :slightly_smiling_face:

@Terdious I wrote an entire spec with Fable 5 for managing sub-containers.

The general idea is that an external integration can declare in JSON the « sub-containers » it needs. In our case, this would allow us, for example, to declare Frigate + Mosquitto directly from the integration.

The spec is available here:

(see the Sub-containers section)

Then, it’s Gladys that handles the entire lifecycle of these containers: creation, startup, shutdown, deletion… And if the integration is deleted, everything is automatically cleaned up (containers + volumes).

For the specific case of Mosquitto, there is an interesting constraint: it needs files present on the disk before it can start (especially for passwords). So, the declared containers can be created but stopped by default, then explicitly started by the integration when it has finished its configuration. An API /start is provided for this.

What do you think?

I launched Fable 5 on 2 projects:

  • Integration type Frigate with sub-containers
  • Integration type « communication » (ex: Telegram)

1st integration on the store!! :partying_face:

If you want to install it:

Then:

I’ll probably work on it and discover all this tonight and tomorrow evening, I’ll give you an update after ^^

And that’s great ^^ I’m going with that, plus we have the HA reference which is ideal for AI :wink:
I switched to Max+ for pro dev this week, and they reset my credits yesterday… I guess as a gift ^^ so I have some leeway until Monday :

Incredible, enjoy it while Fable 5 is out :smiley:

If you want to create dozens of integrations, go for it! Just give the template and SDK references, and the AI should handle it without too many problems.

Then, to test an integration, you have 2 options, and nothing to configure, everything is already ready in the template, it’s turnkey :slight_smile:

Option 1: Test via GitHub

You create a release in one click, and it automatically generates a Git tag + a push of the image to the GitHub Container Registry. Super simple :

Then, in Gladys, you paste the repo link :

Option 2: Test via Docker image

More practical for testing a PR before publishing it in production for all users of the integration.

You start by choosing the branch to build :

This will then publish a Docker image tagged with the branch name.

In the release, you will find all the information to copy into Gladys :

To copy into the interface :

If you encounter any bugs or feedback, let me know! And don’t hesitate to post a thread on the forum for each integration, I’ll check as soon as I can this weekend :muscle: