External integrations unavailable on ARM SBCs without CONFIG_CFS_BANDWIDTH support (Khadas, potentially Raspberry Pi)

Development
1

Hello,

I am reporting a blocking issue for any external integration on my Khadas VIM1S (Ubuntu 24.04, kernel Amlogic 7.0.0+).

Symptom — unable to start an external integration (tested with Shelly), error in the logs:

Cannot restart container XXX: ... error setting cgroup config for procHooks process: 
openat2 /sys/fs/cgroup/.../cpu.max: no such file or directory

Isolated cause, reproducible outside Gladys:

bash

docker run --rm hello-world              # OK
docker run --rm --cpus="1.0" hello-world # fails with the same error

Kernel confirmation :

bash

cat /boot/config-$(uname -r) | grep CFS_BANDWIDTH
→ CONFIG_CFS_BANDWIDTH is not set

This kernel compile flag provides cpu.max (CPU quota cgroup v2). Without it, no --cpus limit can be set on a container — so as soon as external integrations apply this limit at creation, it systematically fails, with no possible workaround on the system config side (tested: cgroup driver systemd and cgroupfs, cgroup delegation verified correct at all levels).

Potential scope : this is not isolated to my hardware. The Raspberry Pi has historically had the same issue (ticket raspberrypi/linux#3387, still open).

Suggestion : make the CPU limit optional, or switch to --cpu-shares (weighting, does not require cpu.max) rather than a strict quota — or detect the absence of cpu.max and create the container without a limit rather than failing.

I remain available to test on this hardware if needed. Thanks

2

Welcome to the forum, @Didier_Hilary!

Doesn’t your problem look like the one we had with Synology?

If so, we’ll turn your post into a feature request to support other devices.

3

Good evening,

The problem doesn’t seem to be exactly the same, the following command gives the correct syntax if I understand the Synology problem correctly!!

khadas@Khadas:~$ docker info --format ‹ {{json .}} › | python3 -m json.tool | grep -i cfs
« CpuCfsPeriod »: true,
« CpuCfsQuota »: true,

Perhaps the problem comes from my system, it doesn’t handle quotas well!!

Is there a way to disable this quota mechanism at the Gladys integration level?
Thanks

Best regards

4

@pierre-gilles have you already seen this issue?

5

I ran Fable on it

6

Hi everyone!

This topic is now under development.

A PR has been opened to start external integrations on kernels without CFS bandwidth support:

Feel free to follow the PR, test (optional, especially for small requests) and give your feedback here if needed.

7

Hi @Didier_Hilary,

I propose an arm64 build for you to test this fix:

ghcr.io/gladysassistant/gladys-preview:claude-external-integrations-arm-unavailable-qk4lsh-arm64

You can test it with this command (be careful with Docker volumes, the port, make sure to modify this command according to your installation):

sudo docker run -d \
  --log-driver json-file \
  --log-opt max-size=10m \
  --cgroupns=host \
  --restart=always \
  --privileged \
  --network=host \
  --name gladys-claude-external-integrations-arm-unavailable-qk4lsh-arm64 \
  -e NODE_ENV=production \
  -e SERVER_PORT=80 \
  -e TZ=Europe/Paris \
  -e SQLITE_FILE_PATH=/var/lib/gladysassistant/gladys-production.db \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /var/lib/gladysassistant:/var/lib/gladysassistant \
  -v /dev:/dev \
  -v /run/udev:/run/udev:ro \
  ghcr.io/gladysassistant/gladys-preview:claude-external-integrations-arm-unavailable-qk4lsh-arm64
8

I also deployed the fix in 5.0.1:

9

Hello,

With version 5.0.1, I managed to install the service on the machine :slight_smile:

But despite this, I hadn’t noticed that my modules are Shelly Gen1, so not integrable in Gladys!!

I discovered another infrastructure problem on my Khadas VIM1S that deserves to be reported, as it potentially affects all external integrations in isolated bridge networks, not just Shelly.

The Symptom

An external integration container (network gladys-integrations, isolated bridge 172.30.0.0/24) cannot reach any LAN device — systematic timeout on exit, while the host itself contacts these same devices without any issues.

The Cause: My Kernel Doesn’t Support Any NAT Backend

bash

docker run --rm --network gladys-integrations curlimages/curl http://<IP_LAN> --max-time 5
# → timeout

Upon further investigation:

  • nftables missing: CONFIG_NF_TABLES is not set in the kernel
  • iptables-legacy also missing: the ip_tables module doesn’t exist either (modprobe: FATAL: Module ip_tables not found)

Therefore, Docker cannot create any NAT/SNAT rules, regardless of the backend — containers in an isolated bridge can never reach the LAN. That’s why "iptables": false was already configured in my daemon.json (probably to allow Docker to start at all, due to the lack of a functional NAT backend) — but this workaround precisely prevents the NAT that external integrations need.

Why This Potentially Affects Other Users

This is a vendor kernel (Amlogic, very stripped-down, originally designed for Android TV boxes) that lacks several standard network/cgroup components. Given the number of lightweight ARM SBCs used in the home automation community (repurposed TV boxes, cheap ARM mini-PCs…), this is probably not an isolated case.

Suggestion

For external integrations that need to contact LAN devices (like Shelly, Tasmota, etc. — anything that isn’t purely MQTT via the host), would it be possible to offer a configuration option in host network mode instead of an isolated bridge? This would completely bypass the need for NAT for users whose system doesn’t support it, with the usual security warning (reduced isolation).

I’m available to test if needed.

10

Hi Didier!

Thanks for this investigation, it’s super precise and very helpful. :folded_hands:

You’ve pinpointed the real cause: your kernel is compiled without netfilter (CONFIG_NF_TABLES is not set, no iptables). Yet it’s netfilter that does the NAT for Docker’s bridged networks — without it, no container in bridge mode can reach the LAN, no matter the application. This isn’t a limitation of Gladys: on this kernel, Docker’s network is broken right under our feet (the official Docker check-config.sh script flags these options as required, by the way).

Regarding network=host, I prefer to be transparent: this won’t be an option in Gladys. The entire security model for external integrations relies on network isolation: an integration can’t reach your machine’s local services, can’t talk to other integrations, and network discovery goes through an explicit contract that you approve during installation. With host mode, all these guarantees disappear — and “with a warning” doesn’t hold up over time: as soon as a popular integration requires it, the warning becomes a reflex click and the promise “an integration can’t do X” is worthless. A detail of importance: without netfilter, container isolation (enable_icc=false) isn’t applied either — so on this kernel, even without host mode, we couldn’t keep our guarantees.

The good news: this is fixable on the OS side, and properly. :tada:

  1. First, check if the modules exist but aren’t loaded: sudo modprobe nf_tables then ls /lib/modules/$(uname -r)/kernel/net/netfilter/. Given your CONFIG_NF_TABLES is not set, it’s unlikely, but it’s worth 30 seconds.
  2. The solution I recommend: switch to Armbian for your VIM1S. There are official, maintained Armbian images for the Khadas VIM1S, with a kernel that includes netfilter. Docker (bridge, NAT, isolation) works normally on it. Don’t forget to back up Gladys first, you’ll restore it on the new installation.
  3. Alternatively, you can report the issue to Khadas: there are already tickets on their build tool about this (fenix #297, fenix #68) — the missing netfilter modules in their vendor kernels are a known issue for them.

Keep me posted if you try Armbian, your feedback will help everyone on SBCs with stripped-down vendor kernels! :blush:

11

Good evening,

I migrated to Armbian and reinstalled Gladys, everything works correctly. I installed the Shelly support and it discovered my Shelly 2.5 Gen1 module, I have the information returned by the module but I cannot control the relays with the integration.

Thanks for the info on Armbian

Best regards

Installed version:

PRETTY_NAME=« Armbian 26.8.3 noble »
NAME=« Ubuntu »
VERSION_ID=« 24.04 »
VERSION=« 24.04 LTS (Noble Numbat) »
VERSION_CODENAME=noble
ID=ubuntu