Snips not working due to problems with audio output

Hi everybody,

in the beginning I’ve tested my snips setup with a usb headset which is recognized as individual audio input and output devices until that worked fine.

I have now tried to decouple the IO devices. Input is now a Matrix IO microphone board and output goes via a speaker connected to the 3mm jack. Input and keyword detection are working fine.
However that brings a problem. I may at maximum hear the keyword confirmation sound once before audio output from snips fails always:

[01:22:48] [Hotword] detected on site default, for model Jarvis
[01:22:48] [Asr] was asked to stop listening on site default
[01:22:48] [Hotword] was asked to toggle itself 'off' on site default
[01:22:48] [Dialogue] session with id 'b412b0d6-1c19-49ee-b7b7-b91f0fa3cf73' was started on site default
[01:22:48] [AudioServer] was asked to play a wav of 41.1 kB with id 'f0fcf28a-78d5-48f3-a19f-6d250826aa2c' on site default
[01:23:14] [Dialogue] session with id 'b412b0d6-1c19-49ee-b7b7-b91f0fa3cf73' was ended on site default. The session was ended because one of the component didn't respond in a timely manner
[01:23:14] [Asr] was asked to stop listening on site default
[01:23:14] [Hotword] was asked to toggle itself 'on' on site default

Have a look at the time it hung after “[AudioServer] was asked to play a wav”. After all this is should just be a simple beep.

I’d say there’s a problem with my sound device, but simply running
aplay pathToSomeWavFile
works totally fine without further parameters.

pi@raspberrypi10:~ $ cat /etc/asound.conf
pcm.!default {
    type asym
    playback.pcm {
        type plug
        slave.pcm "hw:0,0"
    }
    capture.pcm {
        type plug
        slave.pcm "hw:2,0"
    }

pi@raspberrypi10:~ $ cat /etc/snips.toml
........
[snips-audio-server]
mike = "MATRIXIO-SOUND: - (hw:2,0)"
Environment="PA_ALSA_PLUGHW=1"
alsa_playback = "default"
........

Does anybody have an idea what could be the problem?

Do you use the 3.5 jack on the Pi or the matrix?
Regardless of hearing sounds, it looks indeed as if there is something wrong.
Half a minut is way to long, are there no error logs?

Thanks for that hint. Hadn’t thought of the matrix’s audio jack, yet. Unfortunately it didn’t really change anything - other than me getting the feeling that the speakers’ background noises increased.

Since I couldn’t find more detailed logs than given before I stopped the audio-server and ran it interactively on the command with the -v parameter. That’s what it produced:

pi@raspberrypi10:~ $ snips-audio-server  -v
[13:26:22.401512] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.bus -> Ok(None)
[13:26:22.401989] DEBUG:snips_common_cli::cli: conf lookup: snips-common.bus -> Ok(None)
[13:26:22.402071] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.mqtt -> Ok(None)
[13:26:22.402142] DEBUG:snips_common_cli::cli: conf lookup: snips-common.mqtt -> Ok(Some("mqtt.mydomainname:1883"))
[13:26:22.402407] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.mqtt_username -> Ok(None)
[13:26:22.402486] DEBUG:snips_common_cli::cli: conf lookup: snips-common.mqtt_username -> Ok(None)
[13:26:22.402563] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.mqtt_password -> Ok(None)
[13:26:22.402699] DEBUG:snips_common_cli::cli: conf lookup: snips-common.mqtt_password -> Ok(None)
[13:26:22.402829] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.mqtt_tls_hostname -> Ok(None)
[13:26:22.402953] DEBUG:snips_common_cli::cli: conf lookup: snips-common.mqtt_tls_hostname -> Ok(None)
[13:26:22.418338] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.alsa_playback -> Ok(Some("default"))
[13:26:22.418509] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.frame -> Ok(None)
[13:26:22.418584] DEBUG:snips_common_cli::cli: conf lookup: snips-common.frame -> Ok(None)
[13:26:22.418659] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.nomike -> Ok(None)
[13:26:22.418724] DEBUG:snips_common_cli::cli: conf lookup: snips-common.nomike -> Ok(None)
[13:26:22.418800] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.mike -> Ok(Some("MATRIXIO-SOUND: - (hw:2,0)"))
[13:26:22.418952] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.disable_capture -> Ok(None)
[13:26:22.419078] DEBUG:snips_common_cli::cli: conf lookup: snips-common.disable_capture -> Ok(None)
[13:26:22.419203] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.alsa_capture -> Ok(None)
[13:26:22.419321] DEBUG:snips_common_cli::cli: conf lookup: snips-common.alsa_capture -> Ok(None)
[13:26:22.419441] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.hijack -> Ok(None)
[13:26:22.419556] DEBUG:snips_common_cli::cli: conf lookup: snips-common.hijack -> Ok(None)
[13:26:22.419720] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.bind -> Ok(None)
[13:26:22.419937] DEBUG:snips_common_cli::cli: conf lookup: snips-common.bind -> Ok(None)
[13:26:22.420067] DEBUG:snips_common_cli::cli: conf lookup: snips-audio-server.bind -> Ok(None)
[13:26:22.420184] DEBUG:snips_common_cli::cli: conf lookup: snips-common.bind -> Ok(None)
[13:26:22.420394] INFO :snips_audio_server_cli: Starting audio server on mqtt for site default
[13:26:22.420701] DEBUG:snips_audio_portaudio : Attempt 0
[13:26:22.420865] DEBUG:snips_audio_portaudio : Mike source capture started
Expression 'alsa_snd_pcm_hw_params_set_period_size_near( pcm, hwParams, &alsaPeriodFrames, &dir )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 923
Expression 'alsa_snd_pcm_hw_params_set_period_size_near( pcm, hwParams, &alsaPeriodFrames, &dir )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 923
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.front
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround40
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround41
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround50
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround51
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround71
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
Expression 'alsa_snd_pcm_hw_params_set_period_size_near( pcm, hwParams, &alsaPeriodFrames, &dir )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 923
[13:26:22.482731] DEBUG:snips_audio_portaudio : Portaudio: Ok("PortAudio V19-devel (built Jul  9 2019 18:34:00)")
[13:26:22.482940] DEBUG:snips_audio_portaudio : Looking for input device called MATRIXIO-SOUND: - (hw:2,0)
[13:26:22.483045] DEBUG:snips_audio_portaudio : Consider: bcm2835 ALSA: IEC958/HDMI (hw:0,1)
[13:26:22.483191] DEBUG:snips_audio_portaudio : Consider: Dummy: PCM (hw:1,0)
[13:26:22.483465] DEBUG:snips_audio_portaudio : Consider: MATRIXIO-SOUND: - (hw:2,0)
[13:26:22.483699] INFO :snips_audio_portaudio : Use input device: "MATRIXIO-SOUND: - (hw:2,0)"
[13:26:22.483862] DEBUG:snips_audio_portaudio : Start capture: InputSettings { params: Parameters { device: Index(DeviceIndex(2)), channel_count: 1, suggested_latency: 0.017414965986394557, is_interleaved: true, sample_format: PhantomData }, sample_rate: 16000.0, frames_per_buffer: 256, flags:  }
[13:26:22.497258] DEBUG:snips_audio_portaudio : start capture loop
[13:26:22.520881] DEBUG:snips_audio_portaudio : Attempt 1
[13:26:22.621058] DEBUG:snips_audio_portaudio : Attempt 2
[13:26:22.621171] DEBUG:snips_audio_portaudio : Mike initialization looking good
[13:26:22.621943] INFO :snips_broadcast_mqtt_server: Starting broadcasting audio on hermes for site "default"
[13:26:34.834344] INFO :snips_audio_alsa::playback : Using ALSA output Some("default") ("bcm2835 ALSA") for "a87efa7c-602a-4b69-9d1e-bab25d94adc0", WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }
[13:26:45.382201] ERROR:snips_audio_server_hermes  : could not play "a87efa7c-602a-4b69-9d1e-bab25d94adc0"
 -> caused by: ALSA function 'snd_pcm_drain' failed with error 'EIO: I/O error'

^C
pi@raspberrypi10:~ $

Restarted audio service.

pi@raspberrypi10:~ $ sam status

Connected to device localhost

OS version ................... Raspbian GNU/Linux 10 (buster)
Installed assistant .......... HomeControl
Language ..................... de
Hotword ...................... personal_hotword
ASR engine ................... snips
Status ....................... Live

Service status:

snips-analytics .............. (not running)
snips-asr .................... 0.63.3 (running)
snips-audio-server ........... 0.63.3 (running)
snips-dialogue ............... 0.63.3 (running)
snips-hotword ................ 0.63.3 (running)
snips-nlu .................... 0.63.3 (running)
snips-skill-server ........... 0.63.3 (running)
snips-tts .................... 0.63.3 (running)

pi@raspberrypi10:~ $

As mentioned it does SOMETIMES work ONCE. In this example it recognized the hotword, played the first (OK, I’m listening-) sound, but then failed at playing the second sound after hearing the actual command.

pi@raspberrypi10:~ $ sam watch
[12:49:43] Watching on raspberrypi1.mydomainname:1883 (MQTT)
[12:49:45] [Hotword] detected on site default, for model Jarvis
[12:49:45] [Asr] was asked to stop listening on site default
[12:49:45] [Hotword] was asked to toggle itself 'off' on site default
[12:49:45] [Dialogue] session with id 'ed9485c7-0b61-406d-90fe-640829c09d8a' was started on     site default
[12:49:45] [AudioServer] was asked to play a wav of 41.1 kB with id '01037a75-2704-48af-9759-bf2e81eb3210' on site default
[12:49:46] [AudioServer] finished playing wav with id '01037a75-2704-48af-9759-bf2e81eb3210'
[12:49:46] [Asr] was asked to listen on site default
[12:49:48] [Asr] captured text "wetter" in 2.0s
[12:49:48] [Asr] was asked to stop listening on site default
[12:49:48] [AudioServer] was asked to play a wav of 93.1 kB with id 'e3a1b300-6903-4260-827e-9db511917edd' on site default

I’d think the hardware cannot really have a problem as playing sound files and the snips speaker/mic tests are working.

Maybe this topic helps:

This did help indeed. Only the environment variable lines are what I need to ommit. Otherwise the service wouldn’t start. So in the end I guess the asound.conf was the key here.

Thanks a lot for the hint!

1 Like

check this issue.