How to install Snips with Home Assistant on a single Raspberry Pi?


#1

Hello guys,

With big interest I also want to start into my own snips business, raspberry pi kit is already ordered. My goal is to voice control my smart home devices (hue brige, logitech harmony, xiaomi mi robot, etc.) with self built commands.
But now I got issues finding a start. Of course I did a little research. I know that I have to integrate my devices in a smart home system like home assistant and use snips to voice control my home assistant devices. But I don’t know exactly how to make the installation for the home assistant with snips on a single raspberry pi.
Do I first have to install Raspbian as OS or should I directly flash the Hass.IO Image on the SD card? Then integrate snips via the Snips addon for Homeassistant or own installation process for Snips?

It would be awesome if you would enlighten me and give me something like a step by step how-to for the installation process. Until now I only found setup guides for installing snips on an existant home assistant, not both on a single raspi. Surely I am satisfied with links as long as they are useful :wink:

Thanks in advance.

Mantajochen


#2

You don’t have to integrate your hardware to any assistant if you don’t want, I don’t, and control my whole house. Snips works with mqtt, so as long as you code your skills to listen to the topics, you can ping any api you want


#3

Directly flash HassIO and then integrate the AddOn indeed.


#4

Thanks for the replies.
What is the advantage if I would use HA with Snips Addon instead of a standalone snips build? Is it easier to code and link the smart home devices? Do I have more features or is it just for the looks?


#5

Well, since your goal is to voice control your smart home devices, you should first integrate those devices in 1 platform. Home Assistant is great for that.
Then, if you have set that up, add voice control to that platform.

If you only have snips, it would be a lot more effort to integrate all devices.


#6

Thanks for that.
At first I tried it with HASS.IO and the Snips addon, but I didn’t get it to work because of the nlu_engine.json version bug. Later I tried another setup on another PI with a raspbian OS with installed hassbian and snips. The migration from my smart home devices to home assistant was in both cases no problems.

But I couldn’t get snips working with my HA devices. At first I tried to control my hue devices. I installed an assistant with a german pre configured app called “Lichtsteuerung” (engl.: lighting control). The action type from that app is already configured to Home Assistant component.
Now I don’t know how to link that assistant app to my home devices. I tried it with intent_script in the configuration.yaml but as soon as I work with variables like objectLocation it didn’t seem to work. Snips sucessfully captures the text and assign it to the intent, but of course it don’t communicate with the devices. (Output from snips-watch -v: “Session was ended on site default. The session was ended because one of the component didn’t respond a timely manner”)

I think I really got a big understanding problem here, maybe you guys can help me.
I have no snips-skill-server installed or any skills configured. Do I have to? Is this how I link the assistant to my devices or is it only the configuration.yaml I have to edit to get this pre configured assistant working?


#7

I think for problems about Snips and Home Assistant, you are better of here:

I also know a bit about HA, so if you want to share the apprapiate part of your configuration yaml that’s also fine.
You do not have to run the skillserver, that is to say, I do not and have some voice controls enabled


#8

Sure. So this is how I basically configured my configuration.yaml based on the intents that were given in the pre configured app “Lichtsteuerung” (screenshots below).

Assistant:


And here are a few Entity IDs of my hue lights which are assigned to home assistant:
HA_states


#9

maybe interesting for further troubleshooting: so I tried to use 3 following commands:

  1. Turn on lights (german: “Schalte alle Lampen ein”)
  2. Switch of the lights in living room (german: “Schalte Wohnzimmer aus”)
  3. Dim the lights in bedroom to 10 % (german: “Dimme Lampen im Schlafzimmer auf 10 %”)

These are the outputs I get from snips-watch -v:

And here is the output from the HASS log:

It seems that the configured variables don’t work in the entity ID. Is there a need for more configuration? What about the phyton code snippet from the assistant? Or am I doing it completely wrong?
Sorry for the lack of understanding, but the documentation is partially incomplete and a lot of maybe helpful documents are offline. Hope you guys can help me again.


#10

Well I wouldn’t integrate it to home assistant. You could also use NodeRed to control your logic. From there you can reach the hue bridge, Roborock and the Harmony hub. I do the same at home.


#11

So, where is your

snips:

in your configuration.yaml?
Without it, the snips component will not be used and HA will not respond, regardless of your intent_script entries.
Which also explains your “not in an timely manner” error messages


#12

Ops, forgot it to mention in the above output. But of course it is exisiting in my configuration.yaml
Any other ideas?


#13

Hi,

in your intent_script you put objectLocation but in your sam watch the slots is called deviceLocation, did you tried with this?

I just tested with my setup, I’m using Hassio on a RPi and I’ve snips installed on a second RPi, but I’m afraid it doesn’t get the slots. I installed a calculator. I can see in the mqtt logs, that snips understands what I’m asking:

hermes/dialogueManager/sessionStarted {"sessionId":"b136d617-7472-4182-bdca-07157cbba7df","customData":null,"siteId":"default","reactivatedFromSessionId":null}
hermes/asr/startListening {"siteId":"default","sessionId":"b136d617-7472-4182-bdca-07157cbba7df","startSignalMs":null}
hermes/asr/textCaptured {"text":"calcule plus trois plus deux","likelihood":0.7563346,"tokens":[{"value":"calcule","confidence":0.4499691,"range_start":0,"range_end":7,"time":{"start":0.0,"end":0.35064295}},{"value":"plus","confidence":0.5500309,"range_start":8,"range_end":12,"time":{"start":0.35064295,"end":0.48}},{"value":"trois","confidence":1.0,"range_start":13,"range_end":18,"time":{"start":0.48,"end":0.71999997}},{"value":"plus","confidence":1.0,"range_start":19,"range_end":23,"time":{"start":0.71999997,"end":0.96}},{"value":"deux","confidence":1.0,"range_start":24,"range_end":28,"time":{"start":0.96,"end":1.89}}],"seconds":2.0,"siteId":"default","sessionId":"b136d617-7472-4182-bdca-07157cbba7df"}
hermes/asr/stopListening {"siteId":"default","sessionId":"b136d617-7472-4182-bdca-07157cbba7df"}
hermes/nlu/query {"input":"calcule plus trois plus deux","asrTokens":[{"value":"calcule","confidence":0.4499691,"range_start":0,"range_end":7,"time":{"start":0.0,"end":0.35064295}},{"value":"plus","confidence":0.5500309,"range_start":8,"range_end":12,"time":{"start":0.35064295,"end":0.48}},{"value":"trois","confidence":1.0,"range_start":13,"range_end":18,"time":{"start":0.48,"end":0.71999997}},{"value":"plus","confidence":1.0,"range_start":19,"range_end":23,"time":{"start":0.71999997,"end":0.96}},{"value":"deux","confidence":1.0,"range_start":24,"range_end":28,"time":{"start":0.96,"end":1.89}}],"intentFilter":null,"id":"0cd36e71-92bf-482d-b3c1-ab2daccf7461","sessionId":"b136d617-7472-4182-bdca-07157cbba7df"}
hermes/nlu/intentParsed {"id":"0cd36e71-92bf-482d-b3c1-ab2daccf7461","input":"calcule plus trois plus deux","intent":{"intentName":"dbddv01:getAddition","probability":0.92532015},"slots":[{"confidence":1.0,"rawValue":"trois","value":{"kind":"Number","value":3.0},"range":{"start":13,"end":18},"entity":"snips/number","slotName":"firstTerm"},{"confidence":1.0,"rawValue":"deux","value":{"kind":"Number","value":2.0},"range":{"start":24,"end":28},"entity":"snips/number","slotName":"secondTerm"}],"sessionId":"b136d617-7472-4182-bdca-07157cbba7df"}
hermes/intent/dbddv01:getAddition {"sessionId":"b136d617-7472-4182-bdca-07157cbba7df","customData":null,"siteId":"default","input":"calcule plus trois plus deux","asrTokens":[[{"value":"calcule","confidence":0.4499691,"range_start":0,"range_end":7,"time":{"start":0.0,"end":0.35064295}},{"value":"plus","confidence":0.5500309,"range_start":8,"range_end":12,"time":{"start":0.35064295,"end":0.48}},{"value":"trois","confidence":1.0,"range_start":13,"range_end":18,"time":{"start":0.48,"end":0.71999997}},{"value":"plus","confidence":1.0,"range_start":19,"range_end":23,"time":{"start":0.71999997,"end":0.96}},{"value":"deux","confidence":1.0,"range_start":24,"range_end":28,"time":{"start":0.96,"end":1.89}}]],"intent":{"intentName":"dbddv01:getAddition","probability":0.92532015},"slots":[{"confidence":1.0,"rawValue":"trois","value":{"kind":"Number","value":3.0},"range":{"start":13,"end":18},"entity":"snips/number","slotName":"firstTerm"},{"confidence":1.0,"rawValue":"deux","value":{"kind":"Number","value":2.0},"range":{"start":24,"end":28},"entity":"snips/number","slotName":"secondTerm"}]}
hermes/dialogueManager/sessionEnded {"sessionId":"b136d617-7472-4182-bdca-07157cbba7df","customData":null,"termination":{"reason":"timeout"},"siteId":"default"}
hermes/asr/stopListening {"siteId":"default","sessionId":null}
hermes/hotword/toggleOn {"siteId":"default","sessionId":null}

sam watch:

[23:09:44] [Hotword] detected on site default, for model hey_snips
[23:09:44] [Asr] was asked to stop listening on site default
[23:09:44] [Hotword] was asked to toggle itself 'off' on site default
[23:09:44] [Dialogue] session with id 'bd957dbe-95ec-4dcc-a942-c700a83d2841' was started on site default
[23:09:45] [AudioServer] was asked to play a wav of 41.1 kB with id 'e88f7927-8bbc-46c9-a299-3ae017a52d80' on site default
[23:09:45] [AudioServer] finished playing wav with id 'e88f7927-8bbc-46c9-a299-3ae017a52d80'
[23:09:45] [Asr] was asked to listen on site default
[23:09:48] [Asr] captured text "calcule trois plus deux" in 2.0s
[23:09:48] [Asr] was asked to stop listening on site default
[23:09:48] [AudioServer] was asked to play a wav of 93.1 kB with id '55a835fd-e4b9-47c7-99ce-b8ef16ef12ef' on site default
[23:09:49] [AudioServer] finished playing wav with id '55a835fd-e4b9-47c7-99ce-b8ef16ef12ef'
[23:09:49] [Nlu] was asked to parse input "calcule trois plus deux"
[23:09:49] [Nlu] detected intent dbddv01:getAddition with probability 1.000 for input "calcule trois plus deux"
              Slots ->
                  firstTerm -> 3 (confidence: 1.000)
                 secondTerm -> 2 (confidence: 1.000)
[23:09:49] [Dialogue] New intent detected dbddv01:getAddition with probability 1.000
              Slots ->
                  firstTerm -> 3 (confidence: 1.000)
                 secondTerm -> 2 (confidence: 1.000)
[23:09:55] [Dialogue] session with id 'bd957dbe-95ec-4dcc-a942-c700a83d2841' was ended on site default. The session was ended because one of the component didn't respond in a timely manner
[23:09:55] [Asr] was asked to stop listening on site default
[23:09:55] [Hotword] was asked to toggle itself 'on' on site default

My configuration.yaml:

snips:
intent_scripts:
getAddition:
    action:
      - service: notify.telegram
        data:
          message: 'Addittion {{ firstTerm }} plus {{secondTerm }}'

In telegram I only get:

Addition plus

So the slots are not passed to HA.

2018-12-20 00:04:41 INFO (MainThread) [homeassistant.helpers.intent] Triggering intent handler <ScriptIntentHandler - getAddition> 2018-12-20 00:04:41 INFO (MainThread) [homeassistant.helpers.script] Script Intent Script getAddition: Running script 2018-12-20 00:04:41 INFO (MainThread) [homeassistant.helpers.script] Script Intent Script getAddition: Executing step call service

So I’m not sure if the slot name should be the same as in the intent_script.


#14

You were right. Indeed I had to put “deviceLocation” as variables and not “objectLocation”. But I still have some issues with calling more than 1 slot in an intent. Is there an opportunity of building operational slots in the intent_script?
f.ex.

intent_script:
  LightsOn:
    action:
      - service: light.turn_on
        data_template:
	  entity_id: light.{{ deviceLocation }}
	  brightness_pct: '{{ value | int }}'	#<-just operational

Because if I script it like this and say “Hey Snips, turn the lights in Living Room on” the output says that there is an missing slot “value” and therefore the lights won’t turn on. If I would say “Hey Snips, turn the lights in Living Room to 100%” the script works.

In the next few days I am going to rebuild my whole system and I will try again. I know that I am not the only one with problems like that so be sure that I will update this post. For me it is still incomprehensible that there are fully deployed and indeed really nice apps in the snips apps store but no manual on how to use it. Thank you guys so far for the help.


#15

Sorry for the late reply, but just use a default filter in your intent_script. In your app the values are not required so you shouldn’t have to change anything on the snips side.

{{ value| int | default('100') }}