How to speak to Gladys with Snips assistant


#1

Hello everybody,

As many people seems to wait for a Snips integration with Gladys, I can propose you a first version of my module gladys-snips

I’ve made a PR to @pierre-gilles, I hope the way I’ve added some features will be the correct way. :slight_smile:

But you should be able to play with Snips without this PR, you won’t have the answer features through Snips.
And Yes, with Snips you can speak to gladys but Gladys can also answer you.

I let you play with it, and if you find it nice, you can vote on Snips console for the Gladys application.


Solution pour la reconnaissance vocale
Api speech gratuite hors google
Reconnaissance vocale Wit.ai
#2

Top ! Un grand bravo pour le boulot


#3

Thanks a lot !
I’ll play with it asap …


#4

Rho, entre mon boîtier 3D et snips intégré, je vais avoir de quoi m’occuper À mon retour de vacances !!

Merci jean philippe !!


#5

Plus qu’un tuto pour une installation facile :stuck_out_tongue:


#6

Beau projet, très intéressant.

Merci et Bravo pour ton boulot.


#7

En fait coté Tuto, j’ai pas voulu ré-écrire les tutos de Snips, on trouve tout globalement sur leur site, certe parfois un peu fouilli.

J’ai mis les liens dans le README qui me semblait utile pour installer facilement chez soit.

Pour ceux qui utilise tout sur un seul RPI (Gladys + Snips sans satellite), ca devrait être assez simple.


#8

I’ve updated the README, I forgot the dynamic injecton part, without injection, Snips will not understand your rooms or device or other Gladys stuff.


#9

La fonction gladys.room.getAll() ne fonctionne pas pour moi .

Il faut avoir Gladys en 3.11.17 ?


#10

Non, j’ai mis ce prérequis parceque je ne connais pas le numéro de version ou seront inclus les changements que j’ai proposé.
Actuellement, Gladys est en 3.11.6.

Cependant, room.getAll() existe bien dans 3.11.6 si je ne m’abuse.

En revanche, j’ai merdé, j’ai rajouté area.getAll() qui risque de bloquer :confused:

Edit:
Je viens de faire une PR pour ajouter area.getAll()
Toutefois, avec les vacances et @pierre-gilles en France, je ne suis pas sur que les PR soient mergées rapidement.

Pour ceux qui veulent tester vraiment, vous pouvez virer le bout de code qui bloque (dans setup.js) en remplacant:

  .spread((injected, deviceTypes) => {
    slots = [];
    for(var i = 0; i < deviceTypes.length; i++){
      slots.push(deviceTypes[i].tag);
    };
    return [inject({"deviceType": slots}), gladys.area.getAll()];
  })
  .spread((injected, areas) => {
    slots = [];
    for(var i = 0; i < areas.length; i++){
      slots.push(areas[i].tag);
    };
    return [inject({"area": slots}), gladys.user.get()];
  })

par

  .spread((injected, deviceTypes) => {
    slots = [];
    for(var i = 0; i < deviceTypes.length; i++){
      slots.push(deviceTypes[i].tag);
    };
    return [inject({"deviceType": slots}), gladys.user.get()];
  })

En revanche pour room.getAll(), @joeypic tu as quoi comme erreur ?


#11

Non desolé , le problème était bien avec area .

Par contre j’ai cette erreur quand je dis n’importe quelle phrase (celle du module snips gladys) :

0|gladys   | Snips : New message in topic hermes/intent/JeanPhilippe:pushToGladysBrain
0|gladys   | gladys.utils.sqlUnique was not able to find element
0|gladys   | Query = "SELECT id, firstname, lastname, email, birthdate, gender, language, assistantName, preparationTimeAfterWakeUp, role, createdAt, updatedAt FROM user WHERE id = ?;"
0|gladys   | Params = "[null]"
0|gladys   | Unhandled rejection Error: NotFound
0|gladys   |     at /home/pi/gladys/api/core/utils/utils.sqlUnique.js:11:31
0|gladys   |     at tryCatcher (/home/pi/gladys/node_modules/bluebird/js/release/util.js:16:23)
0|gladys   |     at Promise._settlePromiseFromHandler (/home/pi/gladys/node_modules/bluebird/js/release/promise.js:512:31)
0|gladys   |     at Promise._settlePromise (/home/pi/gladys/node_modules/bluebird/js/release/promise.js:569:18)
0|gladys   |     at Promise._settlePromise0 (/home/pi/gladys/node_modules/bluebird/js/release/promise.js:614:10)
0|gladys   |     at Promise._settlePromises (/home/pi/gladys/node_modules/bluebird/js/release/promise.js:693:18)
0|gladys   |     at Promise._fulfill (/home/pi/gladys/node_modules/bluebird/js/release/promise.js:638:18)
0|gladys   |     at /home/pi/gladys/node_modules/bluebird/js/release/nodeback.js:42:21
0|gladys   |     at /home/pi/gladys/node_modules/sails-mysql/lib/connections/spawn.js:120:16
0|gladys   |     at Object.poolfully [as releaseConnection] (/home/pi/gladys/node_modules/sails-mysql/lib/connections/release.js:28:12)
0|gladys   |     at Query._callback (/home/pi/gladys/node_modules/sails-mysql/lib/connections/spawn.js:105:35)
0|gladys   |     at Query.Sequence.end (/home/pi/gladys/node_modules/mysql/lib/protocol/sequences/Sequence.js:96:24)
0|gladys   |     at Query._handleFinalResultPacket (/home/pi/gladys/node_modules/mysql/lib/protocol/sequences/Query.js:144:8)
0|gladys   |     at Query.EofPacket (/home/pi/gladys/node_modules/mysql/lib/protocol/sequences/Query.js:128:8)
0|gladys   |     at Protocol._parsePacket (/home/pi/gladys/node_modules/mysql/lib/protocol/Protocol.js:280:23)
0|gladys   |     at Parser.write (/home/pi/gladys/node_modules/mysql/lib/protocol/Parser.js:73:12)
0|gladys   |     at Protocol.write (/home/pi/gladys/node_modules/mysql/lib/protocol/Protocol.js:39:16)
0|gladys   |     at Socket.<anonymous> (/home/pi/gladys/node_modules/mysql/lib/Connection.js:96:28)
0|gladys   |     at emitOne (events.js:116:13)
0|gladys   |     at Socket.emit (events.js:211:7)
0|gladys   |     at addChunk (_stream_readable.js:263:12)
0|gladys   |     at readableAddChunk (_stream_readable.js:250:11)
0|gladys   |     at Socket.Readable.push (_stream_readable.js:208:10)
0|gladys   |     at TCP.onread (net.js:594:20)

J’arrive vraiment pas a comprendre a quoi c’est du. C’est normal le Params égal a null?

En tout cas
Merci pour le module :slight_smile:


#12

Bien joué @Jean-Philippe! :slight_smile: J’ai hâte de tester ça!

Pour la PR ce sera pas fait tout de suite de mon côté, pas mal d’autres tâches en attente avant cette PR, et surtout c’est les fêtes le moment de souffler un peu :wink:


#13

Salut,

Tu as bien définis le paramètre SNIPS_WAKEWORD ?


#14

I’ve just pushed an update of the module, which add a config view to make the configuration simpler on gladys side.


#15

Bonsoir tout le monde !

Tout d’abord, je vous souhaite une excellente année 2019 ! L’année de Gladys v4 ? :wink:

J’ai passé la journée à installer Snips et tout ce qu’il faut. Je crois que je suis arriver à certaines choses mais il me manque le lien entre Snips et MQTT. En effet, Snips ne parvient pas à s’y connecter :

sam watch

Could not start MQTT client on localhost:1883

-&gt; caused by: Connection refused (os error 111)

Du coup, je suis coincé et je ne sais pas comment débloquer la situation. Mon fichier de configuration a l’air ok :

Fichier de config Snips.toml
[snips-common]

# bus = "mqtt"

mqtt = "localhost:1883"

# audio = ["+@mqtt"]

# assistant = "/usr/share/snips/assistant"

# user_dir = "/var/lib/snips"

## MQTT authentication

# mqtt_username = "gladys"

# mqtt_password = "gladys"

## MQTT TLS configuration

# mqtt_tls_hostname = ""

# mqtt_tls_disable_root_store = false

# mqtt_tls_cafile = ""

# mqtt_tls_capath = ""

# mqtt_tls_client_cert = ""

# mqtt_tls_client_key = ""

[snips-analytics]

[snips-asr]

# no_fst_map = true

# beam_size = 8

# model = "/usr/share/snips/asr"

# audio = ["+@mqtt"]

[snips-asr-google]

# credentials = "/usr/share/snips/googlecredentials.json"

# audio = ["+@mqtt"]

[snips-audio-server]

# frame = 256

bind = "default@mqtt"

# mike = "Built-in Microphone"

# disable_playback = false

[snips-dialogue]

# resources = "/usr/share/snips/dialogue"

# session_timeout = 15

# lambda_timeout = 5

# retry_count = 3

# sound_feedback_disabled_default = false

[snips-hotword]

# model = "/usr/share/snips/hotword"

# hotword_id = "default"

# sensitivity = "0.5"

audio = ["default@mqtt"]

# no_vad_inhibitor = false

# vad_messages = false

[snips-injection]

#asr_model = "/usr/share/snips/asr"

#nlu_engine = "/usr/share/snips/nlu_engine"

#keep_nfirst = 3

[snips-nlu]

[snips-tts]

## Choose one tts provider (defaults to picotts)

# provider = "picotts"

# provider = "makerstts"

# provider = "customtts"

## customtts specific configuration (here configured to use picotts using the en-US language)

## available placeholder variables : %%OUTPUT_FILE%%, %%LANG%%, %%TEXT%%

# customtts = { command = ["pico2wave", "-w", "%%OUTPUT_FILE%%", "-l", "en-US", "%%TEXT%%"] }

Est-ce que quelqu’un a réussi à faire fonctionner Snips et le module MQTT de Gladys (celui du store) ?

Merci d’avance !

G


#16

Salut,

Peux-tu faire un sam status ?


#17

Merci.

Voici le résultat :

pi@gladys:~ $ sam status

Connected to device localhost

OS version ................... Raspbian GNU/Linux 9 (stretch)
Installed assistant .......... SnipsDemoWeatherAssistant
Language ..................... en
Hotword ...................... hey_snips
ASR engine ................... snips
Status ....................... Live

Service status:

snips-analytics .............. 0.60.8 (not running)
snips-asr .................... 0.60.8 (not running)
snips-audio-server ........... 0.60.8 (not running)
snips-dialogue ............... 0.60.8 (not running)
snips-hotword ................ 0.60.8 (not running)
snips-nlu .................... 0.60.8 (not running)
snips-skill-server ........... 0.60.8 (running)
snips-tts .................... 0.60.8 (not running)

J’ai voulu tester la démo avant de me lancer pleinement…

Les logs indiquent un reboot en boucle de chaque service


#18

Effectivement, tu peux poster le résultat d’un sudo tail -50 /var/log/syslog


#19
pi@gladys:~ $ sudo tail -50 /var/log/syslog
Jan  1 21:18:01 gladys snips-hotword[30265]:  -> caused by: Connection refused (os error 111)
Jan  1 21:18:01 gladys systemd[1]: Started Snips Dialogue.
Jan  1 21:18:01 gladys systemd[1]: Stopped Snips Analytics.
Jan  1 21:18:01 gladys systemd[1]: Started Snips Analytics.
Jan  1 21:18:01 gladys systemd[1]: Stopped Snips NLU.
Jan  1 21:18:01 gladys systemd[1]: Started Snips NLU.
Jan  1 21:18:01 gladys snips-dialogue[30268]: ERROR:snips_dialogue: Could not start MQTT client on localhost:1883
Jan  1 21:18:01 gladys snips-dialogue[30268]:  -> caused by: Connection refused (os error 111)
Jan  1 21:18:01 gladys systemd[1]: snips-hotword.service: Main process exited, code=exited, status=1/FAILURE
Jan  1 21:18:01 gladys snips-audio-server[30266]: ERROR:snips_audio_server: Could not start MQTT client on localhost:1883
Jan  1 21:18:01 gladys snips-audio-server[30266]:  -> caused by: Connection refused (os error 111)
Jan  1 21:18:01 gladys snips-nlu[30280]: ERROR:snips_nlu: Could not start MQTT client on localhost:1883
Jan  1 21:18:01 gladys snips-nlu[30280]:  -> caused by: Connection refused (os error 111)
Jan  1 21:18:01 gladys systemd[1]: snips-hotword.service: Unit entered failed state.
Jan  1 21:18:01 gladys snips-analytics[30275]: ERROR:snips_analytics: Could not start MQTT client on localhost:1883
Jan  1 21:18:01 gladys snips-analytics[30275]:  -> caused by: Connection refused (os error 111)
Jan  1 21:18:01 gladys systemd[1]: snips-hotword.service: Failed with result 'exit-code'.
Jan  1 21:18:01 gladys snips-tts[30267]: ERROR:snips_tts: Could not start MQTT client on localhost:1883
Jan  1 21:18:02 gladys snips-tts[30267]:  -> caused by: Connection refused (os error 111)
Jan  1 21:18:02 gladys systemd[1]: snips-dialogue.service: Main process exited, code=exited, status=1/FAILURE
Jan  1 21:18:02 gladys systemd[1]: snips-dialogue.service: Unit entered failed state.
Jan  1 21:18:02 gladys systemd[1]: snips-dialogue.service: Failed with result 'exit-code'.
Jan  1 21:18:02 gladys systemd[1]: snips-nlu.service: Main process exited, code=exited, status=1/FAILURE
Jan  1 21:18:02 gladys systemd[1]: snips-nlu.service: Unit entered failed state.
Jan  1 21:18:02 gladys systemd[1]: snips-nlu.service: Failed with result 'exit-code'.
Jan  1 21:18:02 gladys systemd[1]: snips-audio-server.service: Main process exited, code=exited, status=1/FAILURE
Jan  1 21:18:02 gladys systemd[1]: snips-audio-server.service: Unit entered failed state.
Jan  1 21:18:02 gladys systemd[1]: snips-audio-server.service: Failed with result 'exit-code'.
Jan  1 21:18:02 gladys systemd[1]: snips-analytics.service: Main process exited, code=exited, status=1/FAILURE
Jan  1 21:18:02 gladys systemd[1]: snips-analytics.service: Unit entered failed state.
Jan  1 21:18:02 gladys systemd[1]: snips-analytics.service: Failed with result 'exit-code'.
Jan  1 21:18:02 gladys systemd[1]: snips-tts.service: Main process exited, code=exited, status=1/FAILURE
Jan  1 21:18:02 gladys systemd[1]: snips-tts.service: Unit entered failed state.
Jan  1 21:18:02 gladys systemd[1]: snips-tts.service: Failed with result 'exit-code'.
Jan  1 21:18:02 gladys systemd[1]: snips-injection.service: Service hold-off time over, scheduling restart.
Jan  1 21:18:02 gladys systemd[1]: snips-asr.service: Service hold-off time over, scheduling restart.
Jan  1 21:18:02 gladys systemd[1]: Stopped Snips ASR.
Jan  1 21:18:02 gladys systemd[1]: Started Snips ASR.
Jan  1 21:18:02 gladys systemd[1]: Stopped Snips Injection.
Jan  1 21:18:02 gladys systemd[1]: Started Snips Injection.
Jan  1 21:18:02 gladys snips-asr[30297]: ERROR:snips_asr: Could not start MQTT client on localhost:1883
Jan  1 21:18:02 gladys snips-asr[30297]:  -> caused by: Connection refused (os error 111)
Jan  1 21:18:02 gladys snips-injection[30298]: ERROR:snips_injection: Could not start MQTT client on localhost:1883
Jan  1 21:18:02 gladys snips-injection[30298]:  -> caused by: Connection refused (os error 111)
Jan  1 21:18:02 gladys systemd[1]: snips-asr.service: Main process exited, code=exited, status=1/FAILURE
Jan  1 21:18:02 gladys systemd[1]: snips-asr.service: Unit entered failed state.
Jan  1 21:18:02 gladys systemd[1]: snips-asr.service: Failed with result 'exit-code'.
Jan  1 21:18:02 gladys systemd[1]: snips-injection.service: Main process exited, code=exited, status=1/FAILURE
Jan  1 21:18:02 gladys systemd[1]: snips-injection.service: Unit entered failed state.
Jan  1 21:18:02 gladys systemd[1]: snips-injection.service: Failed with result 'exit-code'.

#20

Je pense que tout vient de la configuration du fichier snips parce que Gladys me dit qu’elle reussit à se connecter à MQTT