Snips-Custom-HotWord not working for Satellite Configuration A


#1

Hi,
I’ve setup a Snips master device and added Psycho’s snips-custom-hotword script, works great thanks very much. Have also setup a Snips satellite device (Configuration A on a Pi Zero + ReSpeaker 2-mic) using the documentation at https://docs.snips.ai/articles/raspberrypi/satellites . Unfortunately the satellite does not respond to the custom or default (‘Hey Snips’) hotwords.

MQTT.fx shows satellite MQTT audio frames being received by the master device but logs indicate no hotword recognition. I’m beginning to suspect the master/satellite ‘configuration A’ is not supported by the snips-custom-hotword script and I’ll need to install this script onto the satellite as well.

Any thoughts appreciated.


#2

Hello,

Can you post the snips.toml of the twice device ?

Ced


#3

Hi Ced,

Thanks for responding.

Master Device - “rasp-m01”

######################################

[snips-common]
# bus = "mqtt"
# mqtt = "localhost:1883"
# audio = ["+@mqtt"]
# assistant = "/usr/share/snips/assistant"
# user_dir = "/var/lib/snips"

## MQTT authentication
# mqtt_username = ""
# mqtt_password = ""

## MQTT TLS configuration
# mqtt_tls_hostname = ""
# mqtt_tls_disable_root_store = false
# mqtt_tls_cafile = ""
# mqtt_tls_capath = ""
# mqtt_tls_client_cert = ""
# mqtt_tls_client_key = ""

[snips-analytics]

[snips-asr]
# no_fst_map = true
# beam_size = 8
# model = "/usr/share/snips/asr"
# audio = ["+@mqtt"]
# partial = false
# partial_period_ms = 250

[snips-asr-google]
# credentials = "/usr/share/snips/googlecredentials.json"
# audio = ["+@mqtt"]

[snips-audio-server]
# frame = 256
bind = "rasp-m01@mqtt"
# mike = "Built-in Microphone"
# disable_playback = false
# disable_capture = false

[snips-dialogue]
# resources = "/usr/share/snips/dialogue"
# session_timeout = 15
# lambda_timeout = 5
# retry_count = 3
# sound_feedback_disabled_default = false

[snips-hotword]
# model = "/usr/share/snips/hotword"
# hotword_id = "default"
# sensitivity = "0.5"
audio = ["rasp-m01@mqtt", "rasp-s01@mqtt"]
# no_vad_inhibitor = false
# vad_messages = false

[snips-injection]
#asr_model = "/usr/share/snips/asr"
#nlu_engine = "/usr/share/snips/nlu_engine"
#keep_nfirst = 3

[snips-nlu]

[snips-tts]
## Choose one tts provider (defaults to picotts)
# provider = "picotts"
# provider = "makerstts"
provider = "customtts"
## customtts specific configuration (here configured to use picotts using the en-US language)
## available placeholder variables : %%OUTPUT_FILE%%, %%LANG%%, %%TEXT%%
# customtts = { command = ["pico2wave", "-w", "%%OUTPUT_FILE%%", "-l", "en-US", "%%TEXT%%"] }

#customtts = { command = ["/home/pi/snipsSuperTTS/snipsSuperTTS.sh", "%%OUTPUT_FILE%%", "mycroft", "%%LANG%%", "FR", "ljm", "FEMALE", "%%TEXT%%", "22050"] }

customtts = { command = ["/home/pi/snipsSuperTTS/snipsSuperTTS.sh", "%%OUTPUT_FILE%%", "amazon", "%%LANG%%", "US", "Joanna", "--", "%%TEXT%%", "22050"] }

################################

Satellite Device - “rasp-s01”

################################

[snips-common]
# bus = "mqtt"
mqtt = "rasp-m01:1883"
# audio = ["+@mqtt"]
# assistant = "/usr/share/snips/assistant"
# user_dir = "/var/lib/snips"

## MQTT authentication
# mqtt_username = ""
# mqtt_password = ""

## MQTT TLS configuration
# mqtt_tls_hostname = ""
# mqtt_tls_disable_root_store = false
# mqtt_tls_cafile = ""
# mqtt_tls_capath = ""
# mqtt_tls_client_cert = ""
# mqtt_tls_client_key = ""

[snips-analytics]

[snips-asr]
# no_fst_map = true
# beam_size = 8
# model = "/usr/share/snips/asr"
# audio = ["+@mqtt"]
# partial = false
# partial_period_ms = 250

[snips-asr-google]
# credentials = "/usr/share/snips/googlecredentials.json"
# audio = ["+@mqtt"]

[snips-audio-server]
# frame = 256
bind = "rasp-s01@mqtt"
# mike = "Built-in Microphone"
# disable_playback = false
# disable_capture = false

[snips-dialogue]
# resources = "/usr/share/snips/dialogue"
# session_timeout = 15
# lambda_timeout = 5
# retry_count = 3
# sound_feedback_disabled_default = false

[snips-hotword]
# model = "/usr/share/snips/hotword"
# hotword_id = "default"
# sensitivity = "0.5"
# audio = ["+@mqtt"]
# no_vad_inhibitor = false
# vad_messages = false

[snips-injection]
#asr_model = "/usr/share/snips/asr"
#nlu_engine = "/usr/share/snips/nlu_engine"
#keep_nfirst = 3

[snips-nlu]

[snips-tts]
## Choose one tts provider (defaults to picotts)
# provider = "picotts"
# provider = "makerstts"
# provider = "customtts"
## customtts specific configuration (here configured to use picotts using the en-US language)
## available placeholder variables : %%OUTPUT_FILE%%, %%LANG%%, %%TEXT%%
# customtts = { command = ["pico2wave", "-w", "%%OUTPUT_FILE%%", "-l", "en-US", "%%TEXT%%"] }

################################


#4

hum, if you used this one : https://github.com/Psychokiller1888/snips-custom-hotword

It’s with snowboy, so it does not use snips-hotword.

If Psycho comes here, maybe he’ll be able to help you :slight_smile:

But if you don’t use snowboy, your config seems correct.

Ced


#5

Snips ASR audio definition is missing. You need to declare there and not under snips-hotword, if you are running the hotword service on your master device only


#6

OK thanks. Yes I am using https://github.com/Psychokiller1888/snips-custom-hotword and thought the ‘audit’ parameter within the master devices [snips-hotword] component is redundant.

So with the hotword service only running on the master device I’ll need to modify its snips.toml such that the audit parameter within [snips-hotword] component is moved to the [snips-asr] component such as: :confused:

[snips-asr]
# no_fst_map = true
# beam_size = 8
# model = "/usr/share/snips/asr"
# audio = ["+@mqtt"]
audio = ["rasp-m01@mqtt", "rasp-s01@mqtt"]
# partial = false
# partial_period_ms = 250


[snips-hotword]
# model = "/usr/share/snips/hotword"
# hotword_id = "default"
# sensitivity = "0.5"
# audio = ["rasp-m01@mqtt", "rasp-s01@mqtt"]
# no_vad_inhibitor = false
# vad_messages = false

Sorry for the dumb questions but still trying to grasp the finer details of snips


#7

Applied the above modification to the [snips-asr] audio parameter on the master device as advised by Psycho, still not working.

Stopped snips-asr on the master and ran the command:

snips-asr -vvv

…with an output of:

[00:04:14.928129] DEBUG:rumqtt::connection                   : snips-asr|4137-rasp-m01-4 Received message on hermes/audioServer/rasp-m01/audioFrame
[00:04:14.928442] DEBUG:rumqtt::connection                   : snips-asr|4137-rasp-m01-4 Received message on hermes/audioServer/rasp-m01/audioFrame
[00:04:14.930776] DEBUG:rumqtt::connection                   : snips-asr|4137-rasp-m01-3 Received message on hermes/audioServer/rasp-s01/audioFrame
[00:04:14.931176] DEBUG:rumqtt::connection                   : snips-asr|4137-rasp-m01-3 Received message on hermes/audioServer/rasp-s01/audioFrame
[00:04:14.933432] DEBUG:rumqtt::connection                   : snips-asr|4137-rasp-m01-3 Received message on hermes/audioServer/rasp-s01/audioFrame
[00:04:14.934021] DEBUG:rumqtt::connection                   : snips-asr|4137-rasp-m01-3 Received message on hermes/audioServer/rasp-s01/audioFrame
[00:04:14.968910] DEBUG:rumqtt::connection                   : snips-asr|4137-rasp-m01-3 Received message on hermes/audioServer/rasp-s01/audioFrame
[00:04:14.972379] DEBUG:rumqtt::connection                   : snips-asr|4137-rasp-m01-3 Received message on hermes/audioServer/rasp-s01/audioFrame
[00:04:14.972991] DEBUG:rumqtt::connection                   : snips-asr|4137-rasp-m01-1 Received message on hermes/audioServer/rasp-m01/audioFrame
[00:04:14.973298] DEBUG:rumqtt::connection                   : snips-asr|4137-rasp-m01-1 Received message on hermes/audioServer/rasp-m01/audioFrame

So it appears audio frames from the satellite snips-audio-server are being forwarded to the master ASR.

Also on the master stopped snips-custom-hotword and ran the following:

sudo /usr/bin/python /home/pi/snips-custom-hotword/customHotword.py /home/pi/snips-custom-hotword/Alice 0.4 -vvv

No indication of hotword hits from the satellite device.

Any words of wisdom appreciated… :confused: