Bluetooth (BLE) sensors: paving the way for external integration

Following an exchange with the creator of Theengs / OpenMQTTGateway, a gap was identified: Gladys does have a Bluetooth integration, but it only covers a handful of devices — whereas the ecosystem of BLE sensors (temperature, humidity, plants, etc.: Xiaomi, SwitchBot, RuuviTag…) is enormous, and projects like Theengs can decode hundreds of them.

For me, this needs to be done through an external integration, not through code in the core. But today, the external integration framework does not provide access to the machine’s Bluetooth, and for good reasons: unlike a Zigbee key (a simple USB device that can be mounted in the container), giving a container access to Bluetooth implies giving it very broad network privileges on the machine, which is incompatible with the store’s security model (third-party integrations that are not audited, installable with one click).

The envisaged approach: a Bluetooth mediator access, on the same model as the network discovery already in place, the core listens to the radio and relays the raw frames, the integration decodes them and publishes sensors and states via the existing APIs. The adapter remains under the control of the core, which will in any case be indispensable when we do Matter/Thread commissioning in BLE.

Note: for those using ESP32 gateways with OpenMQTTGateway, an external integration is already possible today via MQTT, without changing anything in the framework.

The issue here is the use of the Bluetooth adapter of the machine running Gladys.

Hi,

Indeed, if Bluetooth access is possible, it should allow us to decode the frames and make Gladys compatible in read-only mode with all these devices.

Here’s what a decoder like Theengs needs on the API side:

1. It’s the integration that controls the scan — the core executes and arbitrates. Same philosophy as scanNetwork(): the integration requests a bounded scan window, for example gladys.scanBluetooth({ mode, durationSeconds, filters }), and the core maintains control over the adapter. Two nuances compared to scanNetwork:

  • The frames must be streamed during the window (callback as they come), not just returned in bulk at the end; latency matters for states;
  • The window must be renewable (lease model): it’s the integration that decides its cadence — sustained during discovery, relaxed duty cycle in cruise mode (e.g., 10 seconds of scan every 60 seconds), stop when the user disables it.

The core remains the sole arbiter: maximum duration per window, quotas, sharing between integrations, and immediate preemption when Matter/Thread needs the radio.

2. Minimal frame content. To decode, Theengs needs:

  • mac (+ address type public/random) — this is the stable identifier of the device
  • rssi
  • name (local name) if present
  • manufacturerdata (bytes, hex, or base64)
  • servicedata + servicedatauuid
  • serviceUuids announced
  • a timestamp, and the indication of advertising frame vs scan response

A raw field (payload_base64 of the AD structures, consistent with scanNetwork) in addition to the parsed fields would be ideal: it covers exotic cases without freezing the parsing on the core side.

3. Passive AND active scan, at the request’s discretion. Some very common sensors only expose their data in the scan response (thus active scan required). The mode is a parameter of the scan request; the manifest, on the other hand, declares the maximum authorized capacity (e.g., "bluetooth": { "modes": ["passive", "active"] }) that the user grants to the installation.

4. Filtering on the core side, parameterized by the request. Optional but valuable to avoid saturating the WebSocket: filters by MAC prefix, manufacturer_id, or service_uuid passed in the scan options, and a throttle by MAC (e.g., max 1 frame/MAC/second, sensors repeat the same frame in bursts).

5. Adapter management: at the core level, sole owner. This is the point that makes the mediated model really superior to direct access:

  • The core inventories the adapters (hci0, USB dongles…), keeps them up to date (hotplug), monitors their health, and resets them in case of freezing; cheap BLE adapters regularly freeze, a centralized watchdog benefits everyone;
  • The core is the sole owner of each adapter and multiplexes all consumers: core Bluetooth service (GATT/presence), Matter/Thread commissioning, and scan windows of integrations, no more ownership conflicts between processes;
  • On the user side: if multiple adapters are present, it’s in the core config that they assign roles (e.g., hci0 reserved for Matter, USB dongle for integration scans);
  • On the integration side: never a /dev path nor a physical adapter name, at most an optional logical identifier in the scan options when the user has assigned multiple adapters. The integration requests « a scan », not « adapter X ».

Bonus in the long term: multiple adapters assigned to scanning = multiple antennas, better coverage — and the API doesn’t need to change, the core aggregates the frames.

6. User consent. As for existing hardware classes: BLE listening reveals the presence of people (phones, wearables) — an authorization screen during installation, separate from other permissions, seems to be the right granularity.

7. Out of scope for a v1 of the relay (from our point of view): no GATT connection or writing from integrations, advertising in read-only mode already covers the vast majority of sensors, and it keeps the security model simple.

Some resources:

https://www.npmjs.com/package/theengs-decoder

Hi everyone!

This topic is now in development.

A PR has been opened to specify BLE (Bluetooth sensor) advertisement streaming:

Feel free to follow the PR, test (optional, especially for small requests) and give your feedback here if needed.

@1technophile I suggested a spec with Fable 5, what do you think? :slight_smile:

@pierre-gilles Will this pave the way for external OTBR integration?

Thanks, some comments but nothing blocking

Thanks for your feedback @1technophile!

No, I don’t think so!