Platform Update 1.3.0 (0.64.0) - 02/10/2019

#1

Platform Update 1.3.0 (0.64.0) - 02/10/2019

Please update your assistant once your platform is updated. The assistant NLU model needs an update to be compatible with this version.

Added

  • snips-asr, snips-nlu:
    • Added support for Brazilian Portuguese language (pt-br). doc
    • Added a new option thread_number to set the number of concurrent processing thread. doc
  • snips-audio-server:
    • Add support for audio streaming (hermes/audioServer/<site_id>/playBytesStreaming/<chunk_number>/<is_last_chunk>). Allows to use on the fly speech synthesis to improve assistant responsiveness. doc
    • Add portaudio_capture and portaudio_playback on linux to use portaudio audio interface.
  • snips-dialogue
    • Experimental support for site groups and hotword coalescing, see the new options --site-group --sites-in-same-group-default and --hotword-coalescing-duration. doc
  • snips-hotword:
    • Trying to load an unsupported model version will raise a dedicated error message.
  • snips-injection, all-sdk:
    • Add hermes route hermes/injection/onInjectionComplete. Once the injection has completed, the platform wait for the ASR and NLU to reload and then publish an injection complete message. doc
    • Add hermes route to reset injection. This feature allows to clean all previously injected artefacts. doc
    • Add --g2p-model option to provide a path to a g2p model (--g2p-models is now deprecated).
  • snips-nlu, hermes-protocol:
    • Add alternatives intent field to the intent dialogue message for disambiguation. doc
  • snips-watch:
    • Display components readyness. (asr, dialogue, nlu, tts, audio-server and hotword).
    • Display some components errors in snips-watch.
    • audio_playback and dump_audio are now compatible with audio TCP streams: bind a TCP address to a site using the audio option.

Improved

  • snips-asr:
    • A warmup decoding is now performed when loading the model which removes the latency on the first decoding.
    • Change the cache logic. When the size exceeds a threshold, it remains constant. Thus, reduce the memory consumption.
    • Make model loading and usage thread safe. Allows the platform to handle more satellites requests in parallel.
  • snips-audio-server:
    • Change the default frontend used for audio playback and capture from PortAudio to ALSA to mitigate issue #148 (linux only).
  • snips-dialogue:
    • The component responsible of a dialog session timeout is specified in the timeout log instead of The session was ended because one of the component didn't respond in a timely manner.
  • snips-injection:
    • Load g2p resource from assistant by default.
    • Injection service stop if the g2p resources cannot be found.
  • snips-megazord:
    • Made g2p_dir attribute optional when enabling injection. Which allow user to overwrite it instead of forcing to set a path.
  • snips-nlu:
    • Memory usage is dramatically reduced.
    • Improve builtin entity parsing in ES, EN, FR, JA, PT.
    • Accept ambiguous results from DeterministicIntentParser when confidence score is above 0.5.
  • snips-pegasus:
    • Remove package dependency to `snips-platform-common.

Fixed

  • snips-asr:
    • Fix memory leak.
    • Fix race condition type crash at startup on ubuntu 18.04 and Debian Buster. #156.
  • snips-injection:
    • Fix ASR Injection crashes when trying to inject a value containing the symbol #0.
    • Fix memory leaks.
    • Fix crash when injecting a value containing #0.
  • snips-megazord:
    • Fix race conditions in exit by waiting the asr to stop completely before ending the function call.
  • snips-pegasus:
    • Fix broken route pegasus/assistant/infos.
    • Fix broken route pegasus/actions/config not having the correct skill_dir key in the swagger documentation.
    • Fix snips-skill-server action configuration command that was called with the wrong flags order.
    • Fix snips-skill-server action returning an error 500 instead of 200 when an internal error occurs.
    • Fix route /pegasus/actions/config not having the correct skill_dir key in the swagger interface.
  • android-sdk:
    • Fix OpenSLES audio backend related crashes when streaming is not used.

Known issues

  • snips-asr-google: [Won’t fix] snips-asr-google does not work anymore (crashes, hangs). #90
  • snips-audio-server: Plop / Click artefact when playing sound. #198
  • snips-injection: Injection fails when JSON contains an empty value. #192
  • snips-watch: --no-color flag seems to have no effects. #197

[EDIT]
For the ones having audio issues, especially using the embedded sound card on your raspberry pi check this issue

1 Like
#2

This broke my Snips.

ASR training is repeating again and again.

Installed assistant manually, that worked, however the audio server keeps crashing:

sam watch:

[19:17:54] [Audio Server] reported an error on site Wohnzimmer : An error happened while trying to play some audio
[19:17:59] [Dialogue] session with id '7a6ce1c6-33c1-4831-81bd-3a6da945462c' was ended on site Wohnzimmer. The session timed out because the Audio Server component did not respond in a timely manner. Please ensure that the snips audio server is started and running correctly.
[19:17:59] [Asr] was asked to stop listening on site Wohnzimmer
[19:17:59] [Hotword] was asked to toggle itself 'on' on site Wohnzimmer

 $ journalctl -f -u snips-audio-server
-- Logs begin at Thu 2016-11-03 17:16:43 GMT. --
Oct 02 19:07:49 sunnysnips snips-audio-server[1130]: INFO:snips_audio_alsa::playback       : Using ALSA output None ("bcm2835 ALSA") for "214cba97-c71c-4bf4-b796-0b3771440333", WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }
Oct 02 19:08:00 sunnysnips snips-audio-server[1130]: ERROR:snips_audio_server_hermes::player: An error happened while trying to play some audio: could not play "214cba97-c71c-4bf4-b796-0b3771440333"
Oct 02 19:08:00 sunnysnips snips-audio-server[1130]:  -> caused by: an error occured in the audio pipeline: Error("snd_pcm_drain", Sys(EIO))
Oct 02 19:08:00 sunnysnips snips-audio-server[1130]:  -> caused by: ALSA function 'snd_pcm_drain' failed with error 'EIO: I/O error'
Oct 02 19:08:27 sunnysnips snips-audio-server[1130]: INFO:snips_audio_alsa::playback       : Using ALSA output None ("bcm2835 ALSA") for "3d9cb41a-67df-4eb2-ba26-031611e547c6", WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }
Oct 02 19:08:27 sunnysnips snips-audio-server[1130]: INFO:snips_audio_alsa::playback       : Playback for 3d9cb41a-67df-4eb2-ba26-031611e547c6 finished
Oct 02 19:08:32 sunnysnips snips-audio-server[1130]: INFO:snips_audio_alsa::playback       : Using ALSA output None ("bcm2835 ALSA") for "02837ba9-770e-442e-a6c5-33e7059ad2d4", WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }
Oct 02 19:08:42 sunnysnips snips-audio-server[1130]: ERROR:snips_audio_server_hermes::player: An error happened while trying to play some audio: could not play "02837ba9-770e-442e-a6c5-33e7059ad2d4"
Oct 02 19:08:42 sunnysnips snips-audio-server[1130]:  -> caused by: an error occured in the audio pipeline: Error("snd_pcm_drain", Sys(EIO))
Oct 02 19:08:42 sunnysnips snips-audio-server[1130]:  -> caused by: ALSA function 'snd_pcm_drain' failed with error 'EIO: I/O error'

However sam test speaker is working perfectly fine, snips responeses are crippled or nothing :frowning:

 $ sam status

Connected to device sunnysnips

OS version ................... Raspbian GNU/Linux 9 (stretch)
Installed assistant .......... Arysa
Language ..................... de
Hotword ...................... hey_snips
ASR engine ................... snips
Status ....................... Live

Service status:

snips-analytics .............. (not running)
snips-asr .................... 0.64.0 (running)
snips-audio-server ........... 0.64.0 (running)
snips-dialogue ............... 0.64.0 (running)
snips-hotword ................ 0.64.0 (running)
snips-nlu .................... 0.64.0 (running)
snips-skill-server ........... 0.64.0 (running)
snips-tts .................... 0.64.0 (running)

When I talk to one of my satellites (not updated), everything is working fine.

[Audio Server] reported an error on site default : An error happened while trying to play some audio
#3

Hello @HorizonKane, if you did update everything on your rpi you may have update the kernel or the sound driver too , this is a known issue for certain sound configuration , can you check your audio setup and reinstall the audio driver if necessary ? if you are using a snips dev kit : a sam audio setup should be sufficient .

for the backend training issue, could you share your assistant id by PM or in a new thread ?

#4

Hi Marc,

thanks for your reply.

I did not ever install an audio driver on that device. I use a Pi 3 B+ with a PS3 Eye camera as mic. That worked out of the box. As I said, sam test speaker is running fine, only the audio server has that problem. sam audio setup was working but did not solve the issue.

I will PM you my assistant ID.

#5

I solved the audio server issue by doing

sam init

again!

ASR training still infinite loop. It gets done and then starts again and again and again…

#6

Update: The issue came back after I installed the assistant from the .zip.

#7

I have exactly the same issue with ALSA playback (using snips-satellite on Raspberry Pi Rapsbian Stretch) after the upgrade to 1.3.0; all audio is cut off within half a second.

I use the ReSpeaker 4-mics RPi hat, but recording seems to be fine. Also using arecord/aplay on the command-line using the default devices works fine too.

I have no idea what sam init does (I have manually installed the snips-satellite package) and why that would have fixed it, I am curious to know

#8

There is a problem with sound streaming. Often getting errors on my satellites when TTS tries to play sound:

paź 02 23:48:15 kitchen snips-satellite[408]: ERROR:snips_audio_server_hermes : Could not process chunk 25 of steam “92f811e5-e17a-4560-bc88-a139fc854658”: Chunks received in the wrong order: received chunk 25 expected chunk 0
paź 02 23:48:15 kitchen snips-satellite[408]: ERROR:snips_audio_server_hermes : Could not process chunk 26 of steam “92f811e5-e17a-4560-bc88-a139fc854658”: Chunks received in the wrong order: received chunk 26 expected chunk 0
paź 02 23:48:15 kitchen snips-satellite[408]: ERROR:snips_audio_server_hermes : Could not process chunk 27 of steam “92f811e5-e17a-4560-bc88-a139fc854658”: Chunks received in the wrong order: received chunk 27 expected chunk 0
paź 02 23:48:15 kitchen snips-satellite[408]: ERROR:snips_audio_server_hermes : Could not process chunk 28 of steam “92f811e5-e17a-4560-bc88-a139fc854658”: Chunks received in the wrong order: received chunk 28 expected

After that satellite is not responding anymore and only restarting of Snips helps.

#9

That message is from the streaming, it means that the first packet was not received and thus, the subsequent packets are dropped.

But the audio server in the satellite should time out in this case.

Can you open a dedicated thread for this issue? Are you using a specific TTS?

#10

@paulvt @HorizonKane

Regarding the audio-server, we changed the default backend for capture and playback for alsa, you can still use portaudio if you use the options, portaudio_playback and portaudio_capture.

Let’s check that in a dedicated thread.

#11

@paulvt it fixes it on my base not on a satellite (there it would install all Snips services you don’t want on the sat)

Will create a new thread.

#12

Thereis a threadfor the same problem here:[Audio Server] reported an error on site default : An error happened while trying to play some audio

#13

I have created new topic [0.64.0] Streaming problem

#14

@SakanaSan

I’m getting a lot of reports of people not being able to set the volume of snips output since 1.3.0. Alsamixer settings seem not to be taken into account anymore? Any hints?

#15

Hey psycho, thanks for the info,

Do you know which sound cards are impacted?

I m currently fighting with the pi3 embedded sound card.

I noticed some weird ring buffer parameters passed to the playback interface when playing sound with the audio server. ‘/proc/asound/…/hw_parameters’ it can explain why the sound is cut. The lenght of the sound played seems to vary with the sample rate.

aplay seems to handle it better. So I’m trying to figure out a workaround by delegating that to a dmix interface. I still have issues tough.

I am still digging into it. (I wish to have an alsa configuration wizard in my pocket)

#16

Looks like my respeaker 6 circ mic array is not. I know that users with google aiy are for sure.

As reported, on my respeaker I have a feeling like the sound playback is missing something like 0.1 to 0.2 sec, it starts too early in the speech, bit like when you trim your hotword on audacity you take very good care to trim it where there’s no sound wave anymore but when playing it it sounds unatural, like starting too late and ending too early.

#17

This update broke my Snips as well.
Each action is being repeated with each response being repeated also (total of 4 audio responses).

image
image.png1628×982 88.8 KB

#18

@rad,

Seems strange, looks like you have two hotword detector, that triggered two differents dialogue sessions.

#19

Hi,

The release doc give the update command:
apt-get update && apt-get upgrade snips-platform-voice

I guess this is for the master, and will update all component if run on a satellite.

So, is there an official command to update a satellite ?

apt-get update && apt-get upgrade snips-satellite ?

#20

Hum, you are right, we should update that.

On my side I just apt upgrade everything each time :slight_smile: