Platform Update 1.2.1 (0.63.2) - 13/05/2019

#1

New Features

  • snips-satellite: Easy satellite configuration using on demand audio transmission. [doc]
  • snips-pegasus: Snips platform configuration web service (beta). [doc]
  • ios-sdk, android-sdk: IOS and Android SDK support for Dialogue Configure (enable and disable intents).
  • ios-sdk, android-sdk: IOS and Android SDK support for ASR Partials.
  • snips-skill-server: Add cli options to install a skill from GitHib, list skills, delete an assistant related skills, display skills configuration as a json data structure.
  • snips-tts: Add possibility to play a registered wav file through TTS (limited to 10 seconds maximum).

Register the sound:
mosquitto_pub -h my-platform.local -p 1883 -t hermes/tts/registerSound/mysound -f my_wav_file.wav
And use it with the tag [[sound:mysound]] passed within the text string for the TTS to use it
mosquitto_pub -h my-platform.local -p 1883 -t hermes/dialogue/[start|stop|continue]Session -m '{"siteId":"default", "lang":"en", "text": "Do you hear this sound sample ? [[sound:mysound]]. Thanks for listening. ", "id": "someramdomid", "sessionId": "somerandomsessionid"}

Improvements

  • ios-sdk, android-sdk: Removed useless debug logs to improve logs usability.
  • snips-hotword: Updated the default Wake Word model to the latest version in snips-hotword debian package.
  • snips-nlu: Reduce intents false positive by removing confidence normalization when using an intent filter.
  • snips-platform: the services that depended on hermes mqtt will now exit with an error code of 1 if the mqtt connexion to the broker is lost and cannot be restored to prevent unresponsive service.

Fixed

  • ios-sdk: Fix memory leaks.
  • android-sdk: Fix memory leaks when using default TtsHandler. #117
  • snips-skill-server: Skills subfolder is not created automatically with brew install. #108

Known issues

  • snips-asr-google: Google ASR requests hangs randomly. #90
4 Likes
Record Audio files
#2

so for the satellite, it now only needs the snips-satellite application? i can delete hotrod and audio-server ??

I have just the new satellite application installed, altered the toml file to match my information as explained in the docs… and it does not work

just permission denied error

snips-satellite -vvv
[23:31:56.805182] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.bind -> Ok(Some("zero1@mqtt"))
[23:31:56.805527] INFO :snips_satellite      : Using site id : zero1
[23:31:56.805613] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.frame -> Ok(None)
[23:31:56.805689] DEBUG:snips_common_cli::cli: conf lookup: snips-common.frame -> Ok(None)
[23:31:56.805769] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.nomike -> Ok(None)
[23:31:56.805842] DEBUG:snips_common_cli::cli: conf lookup: snips-common.nomike -> Ok(None)
[23:31:56.805916] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.mike -> Ok(None)
[23:31:56.805988] DEBUG:snips_common_cli::cli: conf lookup: snips-common.mike -> Ok(None)
[23:31:56.806071] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.disable_capture -> Ok(None)
[23:31:56.806150] DEBUG:snips_common_cli::cli: conf lookup: snips-common.disable_capture -> Ok(None)
[23:31:56.806234] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.alsa_capture -> Ok(None)
[23:31:56.806308] DEBUG:snips_common_cli::cli: conf lookup: snips-common.alsa_capture -> Ok(None)
[23:31:56.806386] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.hijack -> Ok(None)
[23:31:56.806459] DEBUG:snips_common_cli::cli: conf lookup: snips-common.hijack -> Ok(None)
[23:31:56.806542] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.alsa_playback -> Ok(None)
[23:31:56.806617] DEBUG:snips_common_cli::cli: conf lookup: snips-common.alsa_playback -> Ok(None)
[23:31:56.806701] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.disable_playback -> Ok(None)
[23:31:56.806779] DEBUG:snips_common_cli::cli: conf lookup: snips-common.disable_playback -> Ok(None)
[23:31:56.806857] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.output -> Ok(None)
[23:31:56.806930] DEBUG:snips_common_cli::cli: conf lookup: snips-common.output -> Ok(None)
[23:31:56.807013] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.bind -> Ok(Some("zero1@mqtt"))
[23:31:56.807106] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.bus -> Ok(None)
[23:31:56.807179] DEBUG:snips_common_cli::cli: conf lookup: snips-common.bus -> Ok(None)
[23:31:56.807259] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.mqtt -> Ok(None)
[23:31:56.807331] DEBUG:snips_common_cli::cli: conf lookup: snips-common.mqtt -> Ok(Some("10.0.1.100:1883"))
[23:31:56.807534] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.mqtt_username -> Ok(None)
[23:31:56.807615] DEBUG:snips_common_cli::cli: conf lookup: snips-common.mqtt_username -> Ok(None)
[23:31:56.807699] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.mqtt_password -> Ok(None)
[23:31:56.807775] DEBUG:snips_common_cli::cli: conf lookup: snips-common.mqtt_password -> Ok(None)
[23:31:56.807860] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.mqtt_tls_hostname -> Ok(None)
[23:31:56.807940] DEBUG:snips_common_cli::cli: conf lookup: snips-common.mqtt_tls_hostname -> Ok(None)
[23:31:56.808107] DEBUG:rumqtt::client       : snips-satellite|21264-zero1-1: Client start
[23:31:56.808178] INFO :rumqtt::connection   : snips-satellite|21264-zero1-1: Connection start
[23:31:56.808234] DEBUG:rumqtt::connection   : snips-satellite|21264-zero1-1 new connection
[23:31:56.810829] DEBUG:rumqtt::connection   : Send: Connect(Connect { protocol: MQTT(4), keep_alive: 10, client_id: "snips-satellite|21264-zero1-1", clean_session: false, last_will: None, username: None, password: None })
[23:31:56.814765] DEBUG:rumqtt::connection   : snips-satellite|21264-zero1-1 Received control message Connack(Connack { session_present: false, code: Accepted })
[23:31:56.814876] INFO :rumqtt::connection   : snips-satellite|21264-zero1-1: Connection established
[23:31:56.814935] DEBUG:rumqtt::client       : snips-satellite|21264-zero1-1: Spawning client thread
[23:31:56.815546] DEBUG:snips_audio_portaudio: Attempt 0
[23:31:56.815617] DEBUG:rumqtt::client       : snips-satellite|21264-zero1-1: Entering normal operation loop
[23:31:56.815724] DEBUG:snips_audio_portaudio: Mike source capture started
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.front
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround40
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround41
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround50
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround51
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround71
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm_dmix.c:990:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
[23:31:56.915748] DEBUG:snips_audio_portaudio: Attempt 1
[23:31:57.015941] DEBUG:snips_audio_portaudio: Attempt 2
[23:31:57.116105] DEBUG:snips_audio_portaudio: Attempt 3
ALSA lib pcm_dsnoop.c:583:(snd_pcm_dsnoop_open) unable to create IPC semaphore
ALSA lib pcm_dsnoop.c:556:(snd_pcm_dsnoop_open) The dsnoop plugin supports only capture stream
ALSA lib pcm_dmix.c:990:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
[23:31:57.216278] DEBUG:snips_audio_portaudio: Attempt 4
[23:31:57.316486] DEBUG:snips_audio_portaudio: Attempt 5
ALSA lib pcm_dsnoop.c:583:(snd_pcm_dsnoop_open) unable to create IPC semaphore
ALSA lib pcm_dsnoop.c:556:(snd_pcm_dsnoop_open) The dsnoop plugin supports only capture stream
ALSA lib pcm_dsnoop.c:583:(snd_pcm_dsnoop_open) unable to create IPC semaphore
[23:31:57.416669] DEBUG:snips_audio_portaudio: Attempt 6
[23:31:57.516887] DEBUG:snips_audio_portaudio: Attempt 7
[23:31:57.617068] DEBUG:snips_audio_portaudio: Attempt 8
[23:31:57.693424] DEBUG:snips_audio_portaudio: Portaudio: Ok("PortAudio V19-devel (built May  9 2019 13:17:19)")
[23:31:57.693628] DEBUG:snips_audio_portaudio: Looking for default input device
[23:31:57.693721] INFO :snips_audio_portaudio: Use input device: "seeed-2mic-voicecard: - (hw:1,0)"
[23:31:57.693982] DEBUG:snips_audio_portaudio: Start capture: InputSettings { params: Parameters { device: Index(DeviceIndex(2)), channel_count: 1, suggested_latency: 0.005804988662131519, is_interleaved: true, sample_format: PhantomData }, sample_rate: 16000.0, frames_per_buffer: 256, flags:  }
[23:31:57.717246] DEBUG:snips_audio_portaudio: Attempt 9
[23:31:57.817442] DEBUG:snips_audio_portaudio: Attempt 10
[23:31:57.917609] DEBUG:snips_audio_portaudio: Attempt 11
[23:31:57.968064] DEBUG:snips_audio_portaudio: start capture loop
[23:31:58.017744] DEBUG:snips_audio_portaudio: Attempt 12
[23:31:58.017830] DEBUG:snips_audio_portaudio: Mike initialization looking good
[23:31:58.018342] DEBUG:rumqtt::connection   : Send: Subscribe(Subscribe { pid: PacketIdentifier(1), topics: [SubscribeTopic { topic_path: "hermes/dialogueManager/sessionStarted", qos: AtMostOnce }] })
[23:31:58.018477] DEBUG:snips_common_cli::cli: conf lookup: snips-hotword.hotword_id -> Ok(None)
[23:31:58.018566] DEBUG:snips_common_cli::cli: conf lookup: snips-common.hotword_id -> Ok(None)
[23:31:58.018563] DEBUG:rumqtt::connection   : Send: Subscribe(Subscribe { pid: PacketIdentifier(2), topics: [SubscribeTopic { topic_path: "hermes/dialogueManager/sessionEnded", qos: AtMostOnce }] })
[23:31:58.018783] DEBUG:snips_common_cli::cli: conf lookup: snips-hotword.no_vad_inhibitor -> Ok(None)
[23:31:58.018871] DEBUG:rumqtt::connection   : Send: Subscribe(Subscribe { pid: PacketIdentifier(3), topics: [SubscribeTopic { topic_path: "hermes/hotword/toggleOn", qos: AtMostOnce }] })
[23:31:58.018986] DEBUG:snips_common_cli::cli: conf lookup: snips-common.no_vad_inhibitor -> Ok(None)
[23:31:58.019158] DEBUG:snips_common_cli::cli: conf lookup: snips-hotword.vad_messages -> Ok(None)
[23:31:58.019158] DEBUG:rumqtt::connection   : Send: Subscribe(Subscribe { pid: PacketIdentifier(4), topics: [SubscribeTopic { topic_path: "hermes/hotword/toggleOff", qos: AtMostOnce }] })
[23:31:58.019260] DEBUG:snips_common_cli::cli: conf lookup: snips-common.vad_messages -> Ok(None)
[23:31:58.019347] DEBUG:snips_common_cli::cli: conf lookup: snips-hotword.sensitivity -> Ok(None)
[23:31:58.019424] DEBUG:snips_common_cli::cli: conf lookup: snips-common.sensitivity -> Ok(None)
[23:31:58.019511] DEBUG:snips_common_cli::cli: conf lookup: snips-hotword.model -> Ok(None)
[23:31:58.019586] DEBUG:snips_common_cli::cli: conf lookup: snips-common.model -> Ok(None)
[23:31:58.019667] DEBUG:snips_common_cli::cli: conf lookup: snips-hotword.assistant -> Ok(None)
[23:31:58.019742] DEBUG:snips_common_cli::cli: conf lookup: snips-common.assistant -> Ok(None)
[23:31:58.020147] ERROR:rumqtt::client       : snips-satellite|21264-zero1-1 Disconnected: (Error(SyncMpsc(Disconnected), State { next_error: None, backtrace: None }))
[23:31:58.020155] ERROR:snips_satellite      : Permission denied (os error 13)

[23:31:58.020236] DEBUG:rumqtt::client       : snips-satellite|21264-zero1-1: Entering reconnecting loop
[23:31:58.020298] INFO :rumqtt::client       : not seeking reconnection
[23:31:58.020347] INFO :rumqtt::client       : client thread done
[23:31:58.020398] ERROR:snips_common_cli::cli: Unable to reconnect to the mqtt broker, ending the program.
#3

Hi ozie!

This one is strange, at this point snips-satellite should load the hotword model. It seems that the process does not have the right to do so.

You should have this kind of log instead (taken from my conf at home)

[08:51:29.626868] INFO :snips_hotword_lib::universal: Loading model "/usr/local/etc/assistant/custom_hotword", hotword_key: Some("hey_snips"), model_id: Some("workflow-hey_snips_subww_feedback_10seeds-2018_12_04T12_13_05_evaluated_model_0002")

Can you check the permission policy of your assistant folder and hotword?

To answer your question, yes it is a merge of the hotword and audio-server, it replaces them both on a satellite.

The most basic installation would be to install the following packages only:

sudo apt install snips-hotword-model-heysnipsv4
sudo apt install snips-satellite
#4

Awesome! So a few questions…

  1. Is there any downside to the on demand audio transmission, like a delay?

  2. I was using sam update on my satellites, will this still work with snips-satellite once it’s setup? And any reason there isn’t a “sam init-satelitte” or something like that?

  3. For “actions to be executed on the satellite”, is there an example of this?
    I would like to setup a few simple actions to run locally, like adjusting the volume or responding with the IP address. The only way I could think to do it was setup the skill server on each satellite and bind it to a special name.

#5

Hey,

Here’s few answers,

  1. Any downside? Not that I know of at the moment, only good things for your network usage.
  2. Sam update or init satellite is not planned at the moment, apt is still the way to go now. The idea is nice tough :slight_smile:
  3. Same as on the base, except that, as you said, you need to put some logic to filter out the correct site id.
#6

Regarding #3…
I’m still trying to get my head around this.
So the “code snippet” would run on each device, but just code it to stop if not intended for it?
Or does this have to be done with python scripts, action code, etc…

#7

Code snippet from the console are a very limited way to proceed.

You need to develop more advanced action code to proceed.

The intent message contains the site_id information.

e.g. If you use hermes-python you capture this information in the callback to check if the action code can continue or not on the current device (intent_message.site_id)

Here is an example from the snips-demo-dev-kit repository done by implementing a decorator on the callbacks.

If you are wondering how to retrieve the site_id of each satellite here’s an example from the same project.

#8

Im happy to see next release! I have updated my all satellites with snips-satellite and everything works great so far :slight_smile: Unfortunately my main unit still waits for snips-asr-google fix.

#9

working :slight_smile: …

2 Likes
#10

Up and running on my satellites and hub with no issues. My network thanks you! :slight_smile:

2 Likes