Bluetooth (BLE) sensors: paving the way for external integration

Following an exchange with the creator of Theengs / OpenMQTTGateway, a gap was identified: Gladys does have a Bluetooth integration, but it only covers a handful of devices — whereas the ecosystem of BLE sensors (temperature, humidity, plants, etc.: Xiaomi, SwitchBot, RuuviTag…) is enormous, and projects like Theengs can decode hundreds of them.

For me, this needs to be done through an external integration, not through code in the core. But today, the external integration framework does not provide access to the machine’s Bluetooth, and for good reasons: unlike a Zigbee key (a simple USB device that can be mounted in the container), giving a container access to Bluetooth implies giving it very broad network privileges on the machine, which is incompatible with the store’s security model (third-party integrations that are not audited, installable with one click).

The envisaged approach: a Bluetooth mediator access, on the same model as the network discovery already in place, the core listens to the radio and relays the raw frames, the integration decodes them and publishes sensors and states via the existing APIs. The adapter remains under the control of the core, which will in any case be indispensable when we do Matter/Thread commissioning in BLE.

Note: for those using ESP32 gateways with OpenMQTTGateway, an external integration is already possible today via MQTT, without changing anything in the framework.

The issue here is the use of the Bluetooth adapter of the machine running Gladys.

Hi,

Indeed, if Bluetooth access is possible, it should allow us to decode the frames and make Gladys compatible in read-only mode with all these devices.

Here’s what a decoder like Theengs needs on the API side:

1. It’s the integration that controls the scan — the core executes and arbitrates. Same philosophy as scanNetwork(): the integration requests a bounded scan window, for example gladys.scanBluetooth({ mode, durationSeconds, filters }), and the core maintains control over the adapter. Two nuances compared to scanNetwork:

  • The frames must be streamed during the window (callback as they come), not just returned in bulk at the end; latency matters for states;
  • The window must be renewable (lease model): it’s the integration that decides its cadence — sustained during discovery, relaxed duty cycle in cruise mode (e.g., 10 seconds of scan every 60 seconds), stop when the user disables it.

The core remains the sole arbiter: maximum duration per window, quotas, sharing between integrations, and immediate preemption when Matter/Thread needs the radio.

2. Minimal frame content. To decode, Theengs needs:

  • mac (+ address type public/random) — this is the stable identifier of the device
  • rssi
  • name (local name) if present
  • manufacturerdata (bytes, hex, or base64)
  • servicedata + servicedatauuid
  • serviceUuids announced
  • a timestamp, and the indication of advertising frame vs scan response

A raw field (payload_base64 of the AD structures, consistent with scanNetwork) in addition to the parsed fields would be ideal: it covers exotic cases without freezing the parsing on the core side.

3. Passive AND active scan, at the request’s discretion. Some very common sensors only expose their data in the scan response (thus active scan required). The mode is a parameter of the scan request; the manifest, on the other hand, declares the maximum authorized capacity (e.g., "bluetooth": { "modes": ["passive", "active"] }) that the user grants to the installation.

4. Filtering on the core side, parameterized by the request. Optional but valuable to avoid saturating the WebSocket: filters by MAC prefix, manufacturer_id, or service_uuid passed in the scan options, and a throttle by MAC (e.g., max 1 frame/MAC/second, sensors repeat the same frame in bursts).

5. Adapter management: at the core level, sole owner. This is the point that makes the mediated model really superior to direct access:

  • The core inventories the adapters (hci0, USB dongles…), keeps them up to date (hotplug), monitors their health, and resets them in case of freezing; cheap BLE adapters regularly freeze, a centralized watchdog benefits everyone;
  • The core is the sole owner of each adapter and multiplexes all consumers: core Bluetooth service (GATT/presence), Matter/Thread commissioning, and scan windows of integrations, no more ownership conflicts between processes;
  • On the user side: if multiple adapters are present, it’s in the core config that they assign roles (e.g., hci0 reserved for Matter, USB dongle for integration scans);
  • On the integration side: never a /dev path nor a physical adapter name, at most an optional logical identifier in the scan options when the user has assigned multiple adapters. The integration requests « a scan », not « adapter X ».

Bonus in the long term: multiple adapters assigned to scanning = multiple antennas, better coverage — and the API doesn’t need to change, the core aggregates the frames.

6. User consent. As for existing hardware classes: BLE listening reveals the presence of people (phones, wearables) — an authorization screen during installation, separate from other permissions, seems to be the right granularity.

7. Out of scope for a v1 of the relay (from our point of view): no GATT connection or writing from integrations, advertising in read-only mode already covers the vast majority of sensors, and it keeps the security model simple.

Some resources:

https://www.npmjs.com/package/theengs-decoder

Hi everyone!

This topic is now in development.

A PR has been opened to specify BLE (Bluetooth sensor) advertisement streaming:

Feel free to follow the PR, test (optional, especially for small requests) and give your feedback here if needed.

@1technophile I suggested a spec with Fable 5, what do you think? :slight_smile:

@pierre-gilles Will this pave the way for external OTBR integration?

Thanks, some comments but nothing blocking

Thanks for your feedback @1technophile!

No, I don’t think so!

Hello,

I am developing a tool to communicate via BLE with an old radiation measurement device (FTLab BSG-001). Nothing is standard in its Bluetooth protocol: the device does not send anything on its own and does not publish its measurements in its advertisements. You have to connect via GATT, send it requests, and listen to its notifications. Moreover, it stops broadcasting its advertisements as soon as a client is connected.

Based on point 7 of @1technophile’s message (no GATT connection or writing from integrations in a v1), I understand that external integration is not feasible for this type of device for now. Is that correct?

Two questions:

  • Is GATT access relayed by the core planned for a future version, or is it excluded? I also saw that the specification PR #2935 was automatically closed: is the topic on hold?
  • In the meantime, what approach do you recommend? A small separate Bluetooth service, outside of Gladys, that exposes the measurements in MQTT (or via an API that an external integration would query)?

Hello @ldesgrange,

Yes, you understood correctly: an external integration cannot handle this type of device today, and it won’t be possible in the near future either.

On GATT relayed by the core

This isn’t excluded forever, but it’s not planned. The BLE relay we’re talking about in this topic is only used to listen to announcements, read-only. It’s a deliberate choice. Allowing a one-click installed integration, without re-reading the code, to connect in GATT and write to any Bluetooth device around the house is not the same as listening to announcements in terms of security. There’s also an issue with radio sharing: a GATT connection occupies the adapter. Your device shows this, in fact, as it stops its announcements as soon as a client is connected. The core would need to manage the life of each connection, and this depends a lot on each device.

For PR #2935: it was automatically closed by the bot because it had no more activity. This isn’t a decision to abandon the subject. But we need to be clear: the spec hasn’t been merged and nothing is developed yet, not even the announcement reading part.

In the meantime: yes, a small separate service that publishes in MQTT is the right approach

I advise you to do this:

  • A small independent program that connects in GATT to your BSG-001, sends requests, and listens to notifications. In Python, bleak works very well for this.
  • It publishes the measurements on the MQTT broker with the Home Assistant MQTT Discovery format (homeassistant/sensor/.../config + a state topic). Gladys’ MQTT integration has supported this format since this summer: the device appears automatically in the MQTT discovery tab, without creating anything manually.
  • Another solution: create the device manually in the MQTT integration and publish on gladys/master/device/<external_id>/feature/<feature_external_id>/state.

No need to add an external integration that queries an API: with MQTT, you already have everything you need.

Two remarks:

  • If this program runs on the same machine as Gladys and Gladys’ Bluetooth integration is enabled, the two will fight over the adapter. It’s better to disable Gladys’ Bluetooth integration, use a second dongle, or run the program on another device closer to the meter (a Raspberry Pi, for example).
  • Gladys doesn’t yet have a “radioactivity” category (µSv/h). With Home Assistant discovery, the measurement will be reported as a generic sensor. If you want a real category, we can discuss it in a separate topic, following our criteria for device categories.