@pierre-gilles A proposal from Claude to modify the SDK and the core for managing ONVIF, PTZ… cameras
Proposal: PTZ control for cameras in Gladys
This document proposes adding PTZ control (pan / tilt / zoom) for cameras to Gladys:
the constants to add to the SDK, how a command reaches the integration,
and a control widget on the dashboard.
It is written from an existing external integration — gladys-tapo —
which already communicates ONVIF with TP-Link Tapo cameras and for which PTZ is measured as
available on the camera side, but cannot be expressed on the Gladys side.
1. The Need
A motorized camera (Tapo C210, C500, TC70, and most ONVIF cameras on the market)
can do three things that no Gladys category covers today:
- Move in a direction, faster or slower and further or closer;
- Go to a saved position (« entrance », « garden »);
- Stop.
The corresponding home automation uses are classic:
- « When someone rings the doorbell, the living room camera looks at the entrance »;
- « At night, the camera turns towards the gate; in the morning, it returns »;
- Control the camera manually from the dashboard, without opening the manufacturer’s app.
What Blocks Today
DEVICE_FEATURE_TYPES.CAMERA only contains one entry:
CAMERA: {
IMAGE: 'image',
},
A Gladys camera is therefore, by design, a read-only image source.
No existing category is suitable for bypassing:
| Considered Option |
Why It Is Discarded |
CURTAIN.POSITION for pan |
Displays a roller shutter on a camera; semantically incorrect, and blocks the addition of a real PTZ later |
SWITCH.BINARY by direction |
Four switches for a directional cross; does not carry speed or distance |
| Manifest actions |
Works, but actions are not usable in a scene — yet that’s the main use case |
The purpose of this proposal is therefore to add the missing category rather than to misuse one.
2. Structuring Constraint: setValue Only Carries a Scalar
This is the point that determines the entire design, and it is worth stating before the constants.
A command starts from the interface, goes through the core, and arrives in the integration via
device.setValue:
// server/lib/device/device.setValue.js
async function setValue(device, deviceFeature, value, options = {}) {
const service = this.serviceManager.getService(device.service.name);
await service.device.setValue(device, deviceFeature, value, options);
// ...
}
value is a scalar — a number or a string. However, the requested PTZ control
includes eight parameters:
| Parameter |
Values |
| Pan |
LEFT, RIGHT |
| Tilt |
UP, DOWN |
| Zoom |
ZOOM_IN, ZOOM_OUT |
| Distance |
movement coefficient, from 0 to 1 |
| Speed |
speed coefficient, from 0 to 1 |
| Move Mode |
ContinuousMove, RelativeMove, AbsoluteMove, GotoPreset, Stop |
| Continuous duration |
for ContinuousMove, the duration in seconds before stopping |
| Preset |
the preset token to reach, with GotoPreset |
These eight parameters do not fit into a scalar. Three ways to resolve:
Option A — One Feature per Command, Settings as Device Parameters
Each direction becomes a push type feature, and distance / speed /
continuous duration become device parameters (device.params),
set once during configuration.
camera/ptz-left push → moves left, with device settings
camera/ptz-right push
camera/ptz-up push
camera/ptz-down push
camera/ptz-zoom-in push
camera/ptz-zoom-out push
camera/ptz-stop push
camera/ptz-preset string → the preset token to reach
Pros: Requires no changes to the core. PushDeviceFeature already exists and renders
a button; scenes already know how to trigger a push feature. Immediately
implementable.
Cons: Speed and distance are no longer adjustable by command — a scene cannot say « turn slowly ». Seven features for a single device clutter the list.
Option B — A Single Feature Carrying a Serialized Command
A single camera/ptz feature, of type string, whose value is a JSON:
{ "mode": "ContinuousMove", "pan": "LEFT", "speed": 0.5, "duration": 2 }
Pros: Covers the eight parameters without changing the core — setValue already
accepts strings (and does not persist them, which is suitable: a command is not a state).
Cons: Opaque. The interface cannot build a form from a free string, and the scene editor would display a text field where the user would have to type JSON. This is an API for developers, not for the end user.
Option C — Extend setValue with Named Parameters (Recommended)
setValue already receives an options object that it passes directly to the service. It is sufficient to use it for the secondary parameters, with value carrying the main command.
// The integration receives:
setValue(device, feature, 'LEFT', { speed: 0.5, distance: 0.3, duration: 2 });
Pros: Covers the eight parameters, keeps a readable main value (thus displayable
and scriptable), and introduces no breakage — options exists and is already propagated.
The interface can build a real form, since each parameter is named and typed.
Cons: Requires defining which options are valid per feature type, and that the scene editor knows how to present them.
Recommendation: Aim for C, delivering A as the first step. A is
immediately implementable and already covers « go to position X when Y occurs »,
which is the dominant use case; C then adds fine-tuning without invalidating A.
3. Constants to Add to the SDK
To be added in server/utils/constants.js of Gladys, then reflected in
lib/device-constants.js of the SDK — the latter being, by documented convention at the top of the file, a strict mirror of the former.
3.1 Feature Types
CAMERA: {
IMAGE: 'image',
// --- PTZ: Control of a motorized camera ---
// Directions. The name carries the axis, not the direction of movement expected by the
// protocol: a camera mounted on the ceiling may have an inverted axis, which is
// adjusted in the integration and not in the feature's semantics.
PTZ_LEFT: 'ptz-left',
PTZ_RIGHT: 'ptz-right',
PTZ_UP: 'ptz-up',
PTZ_DOWN: 'ptz-down',
PTZ_ZOOM_IN: 'ptz-zoom-in',
PTZ_ZOOM_OUT: 'ptz-zoom-out',
// Stop a continuous movement. Essential and not just practical:
// a ContinuousMove without Stop leaves the camera rotating until it hits a limit.
PTZ_STOP: 'ptz-stop',
// Saved position to reach. The value is the preset token as the camera names it, never an index: ONVIF cameras return opaque tokens and not an ordered list.
PTZ_PRESET: 'ptz-preset',
// Absolute position, for cameras that know how to report it. Separated from the directions because it is both readable and writable, whereas a direction is just a command.
PTZ_POSITION_PAN: 'ptz-position-pan',
PTZ_POSITION_TILT: 'ptz-position-tilt',
},
3.2 Precedent in Existing Code
The addition follows an already existing pattern: TELEVISION carries LEFT, RIGHT, UP, DOWN,
STOP as distinct feature types, precisely to express a directional cross.
TELEVISION: {
// ...
LEFT: 'left',
RIGHT: 'right',
UP: 'up',
DOWN: 'down',
// ...
},
The proposal therefore does not create a precedent: it applies this to cameras, complementing it with what PTZ requires in addition (presets, stop, absolute position).
3.3 Parameter Values (Option C)
If option C is retained, the coefficients need an explicit domain:
const PTZ_MOVE_MODES = {
CONTINUOUS: 'ContinuousMove',
RELATIVE: 'RelativeMove',
ABSOLUTE: 'AbsoluteMove',
GOTO_PRESET: 'GotoPreset',
STOP: 'Stop',
};
// `speed` and `distance` are coefficients from 0 to 1, deliberately without units:
// a camera expresses its speed in degrees per second, another in motor steps, and
// none documents it. The coefficient is the only portable quantity, and
// integration translates it into what its protocol expects.
const PTZ_COEFFICIENT_MIN = 0;
const PTZ_COEFFICIENT_MAX = 1;
Naming the modes after ONVIF terminology is deliberate: it is the vocabulary of the
standard that almost all cameras implement, and translating it would only add a layer of
correspondence to maintain.
4. Camera Control Widget
4.1 Extend the Existing Widget Instead of Creating a Second One
Gladys already has a camera widget (front/src/components/boxs/camera/Camera.jsx) that
displays the image and, for compatible cameras, the live stream.
A separate PTZ widget would force the user to place two boxes side by side for a
single camera, and to keep them aligned. The proposal is therefore to add the controls
to the existing widget, displayed only if the device has PTZ features.
4.2 Proposed Layout
┌─────────────────────────────────┐
│ │
│ image / live │
│ │
│ ┌───┐ │ ← overlay, bottom right corner
│ │ ▲ │ │
│ ┌───┼───┼───┐ │
│ │ ◄ │ ■ │ ► │ │ ■ = stop
│ └───┼───┼───┘ │
│ │ ▼ │ │
│ └───┘ │
│ [ Enter ▾ ] [-] [+] │ ← presets zoom
└─────────────────────────────────┘
Design points, each motivated:
- Overlay, not below. The camera widget is often placed in a small format; an additional row of buttons below the image would consume the height that is precisely used to see the image.
- Controls hidden by default, revealed on hover (and always visible on touch, where there is no hover). A dashboard viewed at a glance does not need eight buttons permanently.
- Press and hold = continuous movement.
mousedown triggers the direction, mouseup triggers PTZ_STOP. It is the gesture that everyone knows from camera interfaces, and it exactly matches the ContinuousMove / Stop pair.
A single click falls back to a RelativeMove of one step.
- Presets in a dropdown list, not in buttons: their number varies from one camera to another and their names are free.
- Zoom separate from the directional cross, because not all motorized cameras zoom — the buttons only appear if the corresponding features exist.
4.3 Rendering of Features Outside the Widget
Independently of the widget, the PTZ features appear in the « device in a room » view. The routing is done in front/src/components/boxs/device-in-room/DeviceRow.jsx:
const ROW_TYPE_BY_FEATURE_TYPE = {
// ...
[DEVICE_FEATURE_TYPES.CAMERA.PTZ_LEFT]: PushDeviceFeature,
[DEVICE_FEATURE_TYPES.CAMERA.PTZ_RIGHT]: PushDeviceFeature,
[DEVICE_FEATURE_TYPES.CAMERA.PTZ_UP]: PushDeviceFeature,
[DEVICE_FEATURE_TYPES.CAMERA.PTZ_DOWN]: PushDeviceFeature,
[DEVICE_FEATURE_TYPES.CAMERA.PTZ_ZOOM_IN]: PushDeviceFeature,
[DEVICE_FEATURE_TYPES.CAMERA.PTZ_ZOOM_OUT]: PushDeviceFeature,
[DEVICE_FEATURE_TYPES.CAMERA.PTZ_STOP]: PushDeviceFeature,
[DEVICE_FEATURE_TYPES.CAMERA.PTZ_POSITION_PAN]: MultiLevelDeviceFeature,
[DEVICE_FEATURE_TYPES.CAMERA.PTZ_POSITION_TILT]: MultiLevelDeviceFeature,
};
PushDeviceFeature and MultiLevelDeviceFeature already exist: seven of the nine types
therefore require no new component. Only PTZ_PRESET requires one — a dropdown list fed by the camera’s presets — and a PtzControl component grouping the directional cross would be desirable to avoid displaying seven lines of stacked buttons.
Note: a feature whose type is not in this table does not fail, it is rendered as a sensor. PTZ features would therefore be visible but not controllable until the routing is added — which allows delivering the SDK and the interface separately.
5. Usage in Scenes
This is the main interest compared to integration actions, which are not
scriptable.
With option A, a scene « someone rings → the camera looks at the entrance » is written with
the existing action « change the state of a device », setting camera/ptz-preset to the preset token. No new scene action is necessary.
With option C, the scene editor gains by proposing the named parameters (speed,
distance, duration) in the form of this action, rather than leaving them to the device settings.
6. Suggested Breakdown
Each step has its own value and can be delivered alone:
- Constants in the SDK and the core — the
CAMERA.PTZ_* types. No visible effect,
but immediately unlocks integrations: an integration can publish the features
and be controlled by the API, even before the interface knows how to display them.
- Routing in
DeviceRow.jsx — reuses PushDeviceFeature and
MultiLevelDeviceFeature. Makes the features controllable from the device view at a very low cost.
PtzControl component — the grouped directional cross and the preset list.- Integration into the camera widget — the overlay described in 4.2.
- Named parameters (option C) — speed, distance, duration per command, and their presentation in the scene editor.
Steps 1 and 2 are sufficient to make PTZ usable end-to-end, including scenes.
That said, the two aren’t incompatible, with Claude Code on mobile 
.