Widget Photo - integration source issues (Immich)

Feature requests
1

Context

This document specifies how the Photo widget in the dashboard fetches from a photo provider (« photo provider ») rather than from a manually entered list of URLs, and defines the contract that any provider — internal service or external integration — must implement. Immich is the first provider.

The Photo widget (box.type = 'photo') already exists:

  • rendering: front/src/components/boxs/photo/PhotoBox.jsx (slideshow, forward/back navigation, indicators, image memory cache, preloading the next image) and EditPhotoBox.jsx (list of URLs + captions, cropping, interval, displaying captions);
  • model: DASHBOARD_BOX_TYPE.PHOTO = 'photo' (server/utils/constants.js), configuration validated by the Joi schema of server/models/dashboard.js (photos: array of { url, caption }, max 100, photo_fit, photo_slideshow_interval 0–3600, photo_show_caption);
  • image retrieval: GET /api/v1/dashboard/photo/proxy?url= → server/lib/dashboard/dashboard.getPhoto.js. The server downloads the image (so a local NAS remains visible remotely via Gladys Plus), re-encodes it in JPEG 800×400, quality 80 via resizeImageBuffer, and returns the string "image/jpeg;base64,…" that the front consumes in src={data:${image}}.

What’s blocking today. Feeding the widget from an Immich server is impossible without dedicated server code, for three cumulative reasons:

  1. Authentication. Immich requires an x-api-key header on every request. The current proxy does a naked GET, without a header: it cannot structurally talk to Immich.
  2. Network. The current proxy intentionally blocks the local loop and link-local (SSRF protection, the URL coming from a free field). However, a self-hosted Immich is very often reachable at http://localhost:2283, http://immich-server:2283 (Docker network) or on a private IP: the « manual URL » path is the wrong tool for an address configured once by an administrator.
  3. Dynamism. A list of URLs is static. An Immich album grows, and the « memories from that day » change every day: the source must be resolved into a list of photos at runtime, not at configuration.

Guiding Principle: The Core Knows No Provider by Name

The precedent is the weather widget (docs/specs/external-integrations.md §B.18): weather.get enumerates the stateManager and retains any service exposing weather.get(options), the widget optionally pins a provider (GET /api/v1/weather/provider, then ?service=), and a pivot format standardized by the core isolates the UI from each provider’s payloads. No hard-coded getService('openweather').

This specification exactly transposes this model to photos:

  • the core exposes a photo.* capability (gladys.photo) that enumerates services exposing photo.getSources(...) / photo.getPhotos(...) / photo.getImage(...);
  • Immich is one provider among others, implemented in v1 as an internal service (server/services/immich), exactly like openweather is for the weather;
  • an external integration of type: "photo" (Google Photos, PhotoPrism, Synology Photos, Nextcloud Photos…) will be able to implement the same contract without touching the core or the widget (phase 2, §F).

The cost is the same as for the weather: a generic core lib + a service. The benefit is that the 2nd, 3rd, 10th photo source will cost the core nothing.

Assumed Divergence with Weather: No Automatic Mode

Weather has a single « correct » answer (the weather here): the core can therefore try providers in order and take the first one that responds. Photos do not have this property: « my Immich’s Vacations 2019 album » has no equivalent with another provider. Therefore:

  • the widget in provider mode always pins a service (photo_provider) and a source (photo_source_type + photo_source_id);
  • there is no automatic mode, nor silent fallback: if the pinned provider is absent, stopped, or not configured, the widget displays an explicit state (§C.4) rather than someone else’s photos.

Scope

In Scope (v1)

  • Generic « photo provider » contract + core lib gladys.photo + REST routes (§B).
  • Internal service Immich: configuration page (URL + API key + connection test), album listing, album/memories resolution, authenticated image proxy (§D).
  • Photo widget: source mode selection, provider and source selection, order, ceiling, automatic captions (§C).
  • Full backward compatibility of the « manual URLs » mode: no existing widget is modified, no migration.

A. Widget Configuration (Data Model)

A widget’s configuration lives in the JSON of the boxes in t_dashboard: no database migration. Only the Joi schema of server/models/dashboard.js is extended.

Field Type Default Role
photo_source_mode 'manual' | 'provider' 'manual' Source mode. Absent ⇒ 'manual': this ensures backward compatibility of already registered widgets.
photo_provider string (service name) — Pinned provider, e.g., immich. Required if photo_source_mode === 'provider'.
photo_source_type 'album' | 'memories' — Source type with this provider. Free value on the contract side (§B.2), validated by the provider, not by the core.
photo_source_id string ≤ 128 '' Source identifier (Immich album UUID). Empty for a source without an identifier (memories).
photo_order 'recent_first' | 'oldest_first' | 'random' 'recent_first' Display order (§E.2).
photo_max integer 1–100 50 Ceiling of photos loaded from the source (§E.3). Aligned with the .max(100) already applied to photos.
photo_caption_mode 'auto' | 'none' 'auto' In provider mode, caption generated from metadata (§E.4) or no caption.
photos, photo_fit, photo_slideshow_interval, photo_show_caption, name unchanged — photos is only read in manual mode; the others apply to both modes.

Additions to the Joi schema (server/models/dashboard.js):

photo_source_mode: Joi.string().valid('manual', 'provider'),
photo_provider: Joi.string().allow('').max(64),
photo_source_type: Joi.string().allow('').max(32),
photo_source_id: Joi.string().allow('').max(128),
photo_order: Joi.string().valid('recent_first', 'oldest_first', 'random'),
photo_max: Joi.number().integer().min(1).max(100),
photo_caption_mode: Joi.string().valid('auto', 'none'),

The schema remains permissive on photo_source_type (bounded string, not a valid()): adding a favorites source with a provider should not require a core modification, exactly like the type of the external integration manifest is not enumerated by the widget.

B. « Photo Provider » Contract (Core)

New lib server/lib/photo/, mounted in server/lib/index.js under gladys.photo, on the model of server/lib/weather/.

server/lib/photo/
  index.js                 // Photo(service) + prototypes
  photo.getProviders.js    // duck-typed enumeration
  photo.getSources.js      // selectable sources of a provider
  photo.getPhotos.js       // source resolution -> normalized list
  photo.getImage.js        // bytes of a photo -> data URI, with cache
  photo.normalize.js       // normalizeSources / normalizePhotos
  constants.js             // identifier regexes, ceilings, cache TTL

B.1 Enumeration (duck typing, like weather.getProviders)

function getProviders() {
  const serviceNames = this.service.stateManager.getAllKeys('service');
  return serviceNames
    .filter((serviceName) => {
      const service = this.service.getService(serviceName);
      return service && service.photo && typeof service.photo.getSources === 'function';
    })
    .sort();
}

A service is a provider if it exposes the three functions photo.getSources, photo.getPhotos, photo.getImage. getSources acts as a probe (an incomplete provider is a provider bug, not a case to be handled case by case in the core); the core checks the other two at the time of the call and raises NotFoundError if they are missing.

B.2 Pivot format of a source

{
  "type": "album",
  "id": "0d5f4c2e-…-uuid",
  "label": "Vacances 2019",
  "count": 248
}
Field Required Normalization applied by the core
type yes ^[a-z][a-z0-9-]{0,31}$, otherwise the source is discarded
id yes (can be "") ^[A-Za-z0-9._:-]{0,128}$, otherwise discarded. "" = unique source of its type (memories)
label yes string bounded to 100 characters, truncated
count no finite integer ≥ 0, otherwise removed

List bounded to 200 sources; beyond that, truncated (a user does not choose from a menu of 2000 albums — §E.5 handles search).

B.3 Pivot format of a photo

{
  "id": "3f0a…-uuid",
  "caption": "Rome — 12 août 2019",
  "taken_at": "2019-08-12T14:03:11.000Z"
}
Field Required Normalization
id yes ^[A-Za-z0-9._:-]{1,128}$, otherwise the photo is discarded. This is the opaque token that the widget will return to photo.getImage — the core never interprets it
caption no string bounded to 200 characters, truncated; empty ⇒ removed
taken_at no valid ISO date, otherwise removed

No URL appears in the pivot, by design: the browser must never join the provider directly (no IP leak, no API key exposed, no remote access Gladys Plus break). Sorting and the ceiling are applied by the core after normalization (§E.2, §E.3), so that all providers behave the same.

B.4 Format of an image

photo.getImage returns the string "image/jpeg;base64,…" — exactly the format already produced by dashboard.getPhoto and consumed by PhotoBox/EditPhotoBox in data:${image}. The provider, on the other hand, returns a Buffer: it’s the core that validates and re-encodes (§B.6), so that validation never depends on the provider.

B.5 REST Routes

Added in server/api/routes.js via a photo.controller.js (the model is weather.controller.js), all authenticated: true without admin: any user configures their dashboard, as for GET /api/v1/weather/provider. The payload contains nothing operational (no server URL, no key).

Route Parameters Response
GET /api/v1/photo/provider — [{ "service_name": "immich", "label": "Immich" }] — same shape as weather providers (label = display name of the manifest for external integration, null for an internal service, the front i18n taking over)
GET /api/v1/photo/source service (required) [{ type, id, label, count }] — §B.2
GET /api/v1/photo/list service, source_type, source_id, order, limit { "photos": [{ id, caption, taken_at }] } — §B.3
GET /api/v1/photo/image service, photo_id "image/jpeg;base64,…" (text/plain, like the existing proxy)

GET /api/v1/dashboard/photo/proxy remains unchanged: this is the path of the manual mode, with its SSRF protection, and it is not concerned by these routes.

Errors: standard Gladys format (errorMiddleware). Unknown or non-provider service → 404 NOT_FOUND. Unconfigured provider → ServiceNotConfiguredError (the front displays the call to action « configure Immich »). Third-party failure → 400 with ERROR_MESSAGES.REQUEST_TO_THIRD_PARTY_FAILED, the same code that the weather widget already knows how to present.

B.6 What the core never trusts

Like normalizeWeather for weather, everything that comes from a provider is normalized and bounded before entering the core:

  • bounded lists (200 sources, 100 photos), whitelisted fields (any unknown field is removed), truncated strings, validated dates, identifiers filtered by regex;
  • image validated on the decoded bytes: only JPEG / PNG / WebP / AVIF / GIF magic numbers (no trust in the third-party’s Content-Type), size ≤ 25 MB (aligned with MAX_SOURCE_IMAGE_BYTES), then systematic re-encoding by resizeImageBuffer to 800×400 JPEG q80. A provider cannot therefore serve an SVG, an HTML, or a 200 MB file through the Gladys origin;
  • photo_id re-checked by the core before any provider call: an identifier outside the regex returns 404 without a single byte being sent to the provider.

B.7 Caches (bounded, in memory)

Cache Key TTL Max Size
Sources service 5 min 1 entry per provider
Photo list service + source_type + source_id + order + limit 5 min 20 entries (LRU)
Image service + photo_id 10 min 60 entries (LRU) — same order of magnitude as the weather image cache (10 min)

After re-encoding, an image weighs ~30–60 KB: 60 entries ≈ 3 MB, acceptable on a Raspberry Pi. The image cache of the front (imageCache of PhotoBox) is also bounded to 60 LRU entries as part of this work: today it grows without limit, which worked with 100 manual URLs but deserves a limit as soon as an album is periodically refreshed.

The random mode is excluded from the list cache or, more simply, drawn on the server side with a seed derived from the cache window: two consecutive loads less than 5 minutes apart therefore return the same order — this is intentional, otherwise the slideshow’s forward/back navigation would jump from one photo to another without coherence.

C. Front

C.1 Editing the widget (EditPhotoBox.jsx)

A first select Photo Source: Manual URLs (default) / From an integration.

  • Manual URLs: the current screen, identical.
  • From an integration:
    1. select Provider ← GET /api/v1/photo/provider. If the list is empty: help block « No photo integration is installed » with a link to the integrations catalog.
    2. select Source ← GET /api/v1/photo/source?service=…, grouped by type: *Albums*, *Souvenirs*), labellabel+countwhen present. Writephoto_source_type**and**photo_source_id` in a single action.
    3. Select Order (recent first / oldest first / random), Maximum number of photos field (1–100, default 50), Automatic captions switch.
    4. Preview : the first 3 resolved photos, loaded via GET /api/v1/photo/image — same role as the PhotoPreview in manual mode (check before saving), without duplicating its debounce logic since there is no character-by-character input.

Common options (cropping, interval, caption display, widget name) remain displayed in both modes.

C.2 Execution (PhotoBox.jsx)

PhotoBox gains a resolution step before its slideshow; everything that follows (current index, transitions, buttons, indicators, preloading, cache) is reused as is.

  • manual mode: photos comes from the configuration, images via /api/v1/dashboard/photo/proxy?url= — unchanged.
  • provider mode: on mount, GET /api/v1/photo/list → list of { id, caption, taken_at } in state; each image via GET /api/v1/photo/image?service=…&photo_id=…, the cache key being the URL of the request. The preloading of the next image works identically.

The component thus works on a resolved list common to both modes; this is the only real internal refactor, and it simplifies getDerivedStateFromProps (which today bounds the index based on props only).

C.3 Refresh

  • On mount of the widget, and with each change in the source configuration.
  • Periodically, every 60 minutes (PROVIDER_LIST_REFRESH_MS) : sufficient for an album that is enriched, and this catches the midnight transition of memories in less than an hour.
  • On return to index 0 of the slideshow if the list is more than 60 min old: a dashboard left on for a wall remains up to date without an additional clock.
  • A refresh that returns a shorter list than the current index brings the index within bounds (existing rule); an identical list does not trigger any image reload, the cache doing its job.

C.4 Display States

Situation Render
Empty source (empty album, no memories today) Explicit empty state, not an error: « No photos in this source today. » (dedicated i18n key, distinct from emptyPhotos)
Unconfigured provider Message + link to the integration configuration page
Missing / stopped / unreachable provider Error message with the provider name pinned — never fallback to another provider
Failure of an isolated image Current behavior: error icon on this photo, the slideshow continues

F. Phase 2 — External integrations of type: "photo" (design, not implemented)

Direct transposition of docs/specs/external-integrations.md §B.18 (weather) and §B.15 (communication). Nothing specified in §A–E changes; the implementation of this phase will update external-integrations.md in the same diff, as its rule requires.

  • Manifest : type: "photo". Installation screen with a dedicated information line (« this integration can provide the photos displayed on your dashboards »).
  • Proxy service : exposes photo.getSources / photo.getPhotos / photo.getImage, relayed via WebSocket:
Command Payload Ack (command-result) Timeout
external-integration.photo.get-sources { message_id } data.sources 15 s
external-integration.photo.get-photos { message_id, options: { source_type, source_id, limit } } data.photos 15 s
external-integration.photo.get-image { message_id, photo_id } data.image (raw base64, no data-URI prefix) 15 s

The 15 s timeout is the same already accepted for camera.get-image and weather.get (third-party API call). The order and limit remain applied by the core: the integration receives limit as an indication (not to transfer 5000 entries), the core re-truncates after normalization.

  • No « device » surface : like weather and communication, a photo integration has no Devices screen, no discovery, no states — only Configuration / Supervision / Logs.
  • Nothing new to validate : normalizeSources / normalizePhotos / the image validation of §B.6 are written to be applied to all providers from v1 — the internal Immich service is treated with the same distrust as a third-party integration, ensuring that phase 2 adds no surface of trust.
Intégration Immich