Gladys Music: ask Gladys to play any music on any speaker

[guim31]

Hi everyone!

What I would like to be able to do, quite simply: « Gladys, play the Daft Punk album in the living room », or in a scene, « At 7 a.m., play my morning playlist in the kitchen ». And that it works no matter the music source (my Navidrome, Spotify, Plex, a web radio…) and no matter the speaker (Sonos, amplifier, Chromecast…).

Today, all the building blocks exist, but they don’t talk to each other. Hence this proposal, which I call Gladys Music for now. @pierre-gilles, I’d love your opinion before going further.

Where we are at

• On the speakers side, there’s already a lot to do: Sonos, AirPlay and Google Cast in the core, and external integrations for DLNA, Denon / Marantz amplifiers, MusicCast, LinkPlay (WiiM…), Lyrion (Squeezebox), Echo, VLC…
• On the music side, same thing: Subsonic / Navidrome, Spotify, Plex, Jellyfin, DLNA servers, web radios…
• But in between, nothing. The music features of the core know how to play, pause, next, volume and « play a notification », but not « play that ». And Gladys’ AI has no tool for music: it turns on the light, but it doesn’t know how to play an album.

Result, each integration tinkers in its corner. The DLNA one, for example, knows how to search for an album on a server and play it on an amplifier, but only between DLNA devices, and only from a scene. The subject is not new: there was already « Music service management » in 2022.

The proposal: three blocks in the core

1. Music sources. An integration (from the core or external) can declare itself as a « source »: you give it text and a type (title, album, artist, playlist, radio), it returns results with their cover and how to play them, most often a stream address. Subsonic, Plex, Jellyfin, DLNA servers or a web radio integration fit naturally into this.

2. Players that know how to « play that ». A new music feature, for example play_media, next to play_notification which already knows how to play an address. The core makes the link between the two:

• in the general case, the source provides a stream address, and any player that knows how to play an address plays it (DLNA, Sonos, AirPlay, Cast…);
• when a source only plays at home (Spotify on its Spotify Connect devices), the core only offers compatible players.

**3. One way to use it everywhere:

• a music tool for the AI: « play what, where » (with a default preferred source), plus pause / next / volume. This is what makes the voice command natural, from the phone’s microphone in the app;
• a scene action « Play music » with a search, valid for all sources and all speakers;
• a « Music » widget: a search, a choice of speaker, and off you go.

For external integrations, it would just take an extra declaration in the manifest and the SDK (« I am a source », « this player knows how to play an address »). No need to install another server, unlike Music Assistant in Home Assistant: it would work with what people already have.

What I can contribute

If the idea suits you, I can:

• write the specification, based on the model of docs/specs/external-integrations;
• do the SDK part for external integrations;
• connect the first sources and the first players on my side (Subsonic, DLNA, a web radio integration…) to validate everything;
• and propose a PR for the core, if you prefer I take care of it rather than you doing it yourself.

My questions

1. Does the core seem to you the right place, or do you see it differently?
2. Rather a new feature play_media, or extend play_notification which already knows how to play an address?
3. On the AI side: does a dedicated music tool suit you, alongside those that already control the devices?
4. Does the name « Gladys Music » speak to you, or do you have a better one?

And for the others: does this correspond to your uses? What would you be missing?