Connect snips to hass.io mqtt broker


#1

Hi there, I could use some help from the community.

I’ve decided to build my own voice assistant using Snips and Home Assistant.

The great thing about Snips is how easy it is to set up a simple working voice interface. But on the other hand I don’t really understand what’s happening.

I have set up Snips on a dedicated Pi3 and got it working quickly. Next step I wanted to try was connecting it to home assistant by pointing Snips to my hass.io mqtt broker.

So as a first step I added the broker and credentials in snips.toml. But now I don’t see a reaction anymore. I get no response to the keyword. sam watch logs: listening on 10.0.0.100:1883 and nothing else. Also nothing in my mqtt broker logs.

What am I missing here?


#2

There a couple of things you could check:

  • Have you restarted all Snips services after the change in snips.toml?
  • Have you specified the correct authentication settings for MQTT in snips.toml if your broker requires a user and password?
  • Have a look at the logs with tail -f /var/log/syslog while you are talking to Snips.

#3

Thanks for your response

Snips was properly restarted and mqtt credentials should be okay.

However, in now noticed that I do get a response to hey snips. Only it has a delay of several minutes.

$ sam watch
[12:38:28] Watching on 10.0.0.100:1883 (MQTT)
[12:40:22] [Hotword] detected on site default, for model hey_snips

Syslog:

Dec 31 12:40:22 raspberrypi snips-hotword[1244]: INFO:snips_hotword_hermes        : Hotword detected: site_id:default model:hey_snips (133291ms ago)

Why does this take so long?


#4

That’s really weird, I haven’t seen that behaviour yet… And the delay isn’t there when you use the local MQTT broker on the Snips machine?


#5

No, it works great when I use the local broker

I wonder what is supposed to happen. Is the response routed completely through the remote mqtt broker or should I expect am audio response same as when I use the local broker?

What exactly gets sent to the broker? The intent? The response? Even with the delay I see nothing on the
hass.io broker end.


#6

By using the MQTT from hassio, all your sounddata goes over the network.

You should bridge your snips, so do not use the mqtt from the hassio


#7

This might help you:



#8

Oh interesting, thank you. That’s what I wondered. Sound data means the transcribed text or the actual audio? Might that explain the delay?

What do you mean by bridge your Snips?


#9

The actual audio

You need everting local, but specific topics send to hassio.
Read that here: https://github.com/SkyHyperV/snips-ha-mqtt/blob/master/mosquitto.conf_for_snips

If you set the mqtt to your hassio broker, all will go over the network, including actual audio.
Probably why it takes so long :wink:

Bridging is sending only the topics needed to hassio and keep the rest local


#10

Awesome! Thank you for the resources. Will to check that out later!


#11

Again thank you for the link and the explanation. I have created the bridge to my hass.io broker and I now receive the messages on the home assistant side.

However now my regular Snips Apps do not react anymore. The hotword is detectet, the intent is evaluated, but I get a timeout message and no response on the speaker.
the session was ended because one of the component didn't respond in a timely manner

Any suggestions for me?


#12

Do you have snips enabled on your hassio?
And intent scripts? Because Hassio is the one which should respond now to your snips.


#13

Oh, well, okay. I assumed I could do both. But that’s fine. Thank you for your help!


#14

Maybe you could do both, I don’t know really :smiley:


#15

Hi,

in my setup, both work.
Here my /etc/mosquitto/conf.d/mqtt.conf file on the snips server.

#listener 1883 jf26-pi3
#listener 1883 127.0.0.1
connection_messages true
#log_type all
allow_anonymous true
#password_file /etc/mosquitto/pwfile

connection snipsmqtt
address HASSIOMQTTSERVER:1883
remote_username USERNAME
remote_password PASSWORD
remote_clientid snips
start_type automatic
topic hermes/dialogueManager/# in
topic hermes/asr/# in
topic hermes/hotword/# out
topic hermes/audioserver/# in 

topic hermes/intent/# out
topic hermes/asr/# out
topic hermes/hotword/# out
topic hermes/nlu/# out

Local snips apps are working as well as HA intent_scripts.


#16

Indeed! I have removed the listener lines from the configuration and now both work fine. I didn’t really understand what I needed them for anyway. The video wasn’t very clear there.

Awesome, everything on track now! Thank you!


#17

You can do both and just have them respond to different intents. Should be working fine.


#18

Hi all,

i have been following this thread to setup my snips master with Hassio as well.
As i was having similar issues as described here i have checked my configuration in regards to what you have done but its still not working at all.

Snips is detecting the hotword but it is not reacting or recognizing what is said afterwords:

[21:53:24] Watching on localhost:1883 (MQTT)
[21:53:32] [Hotword] detected on site master, for model hey_snips
[21:53:32] [Asr] was asked to stop listening on site master
[21:53:32] [Hotword] was asked to toggle itself 'off' on site master
[21:53:32] [Dialogue] session with id '180599bc-717e-412a-9a46-6b0cba3b9f08' was started on site master
[21:53:32] [AudioServer] was asked to play a wav of 41.1 kB with id '29896552-03ce-47e0-a477-7359f949fe83' on site master
[21:53:32] [AudioServer] finished playing wav with id '29896552-03ce-47e0-a477-7359f949fe83'
[21:53:32] [Asr] was asked to listen on site master
[21:53:48] [Dialogue] session with id '180599bc-717e-412a-9a46-6b0cba3b9f08' was ended on site master. The session was ended because one of the component didn't respond in a timely manner
[21:53:48] [Asr] was asked to stop listening on site master
[21:53:48] [Hotword] was asked to toggle itself 'on' on site master 

My snips.toml (just the parts i have changed):

[snips-common]
# bus = "mqtt"
mqtt = "localhost:1883"
# audio = ["+@mqtt"]
# assistant = "/usr/share/snips/assistant"
# user_dir = "/var/lib/snips"

[snips-audio-server]
# frame = 256
bind = "master@mqtt"
# mike = "Built-in Microphone"
# disable_playback = false
# disable_capture = false

My mosquitto.conf on snips:

connection_messages true
#log_type all
allow_anonymous true
#password_file /etc/mosquitto/pwfile

connection snipsmqtt
address 192.168.1.104:1883
remote_username xxxxxxx
remote_password xxxxxxx
remote_clientid snips
start_type automatic
notifications true
cleansession true
bridge_protocol_version mqttv311

# default setting but we explicitly set it here
try_private true

topic hermes/dialogueManager/# in
topic hermes/asr/# in
topic hermes/hotword/# out

topic hermes/intent/# out
topic hermes/asr/# out
topic hermes/hotword/# out
topic hermes/nlu/# out

topic hermes/tts/say in
topic hermes/audioServer/# in
topic hermes/tts/sayFinished in

I am seeing the bridge seems to be working as i can see the connection on hassio side:

1547844688: Received PINGREQ from snips
1547844688: Sending PINGRESP to snips
1547844705: Received PUBLISH from snips (d0, q0, r0, m0, 'hermes/hotword/default/detected', ... (214 bytes))
1547844705: Received PUBLISH from snips (d0, q0, r0, m0, 'hermes/asr/stopListening', ... (70 bytes))
1547844705: Received PUBLISH from snips (d0, q0, r0, m0, 'hermes/hotword/toggleOff', ... (70 bytes))
1547844706: Received PUBLISH from snips (d0, q0, r0, m0, 'hermes/asr/startListening', ... (91 bytes))
1547844721: Received PUBLISH from snips (d0, q0, r0, m0, 'hermes/asr/stopListening', ... (36 bytes))
1547844721: Received PUBLISH from snips (d0, q0, r0, m0, 'hermes/hotword/toggleOn', ... (36 bytes))
1547844726: Received PUBLISH from snips (d0, q0, r0, m0, 'hermes/hotword/default/detected', ... (214 bytes))
1547844726: Received PUBLISH from snips (d0, q0, r0, m0, 'hermes/asr/stopListening', ... (70 bytes))
1547844726: Received PUBLISH from snips (d0, q0, r0, m0, 'hermes/hotword/toggleOff', ... (70 bytes))
1547844727: Received PUBLISH from snips (d0, q0, r0, m0, 'hermes/asr/startListening', ... (91 bytes))

Sam status:

OS version ................... Raspbian GNU/Linux 9 (stretch)
Installed assistant .......... HA
Language ..................... de
Hotword ...................... hey_snips
ASR engine ................... snips
Status ....................... Live

Service status:

    snips-analytics .............. 0.60.10 (not running)
    snips-asr .................... 0.60.10 (not running)
    snips-audio-server ........... 0.60.10 (running)
    snips-dialogue ............... 0.60.10 (running)
    snips-hotword ................ 0.60.10 (running)
    snips-nlu .................... 0.60.10 (running)
    snips-skill-server ........... 0.60.10 (running)
    snips-tts .................... 0.60.10 (running)

Do you have any ideas what i could try?

Best Regards