[newbie] adding more stuff to a simple action

I have a very simple action-script working to which I now would like to add extra stuff e.g.
I’d like it to play an mp3-file with the command: mpg321 nice_musical_fragment.mp3
I put the mp3 file in the skills directory and then tried to add the command in the existing action-file but the mp3-file is not played. It plays from the command line with the above mentioned command but I don’t know how to integrate the command in the action-file. Where exactly should I place the command? Can anyone here help me further?

I put the command in the code below in different places but I never got the expected result:

 def action_wrapper(hermes, intentMessage, conf):
    if len(intentMessage.slots.house_room) > 0:
        house_room = intentMessage.slots.house_room.first().value
    if len(intentMessage.slots.on_off) > 0:
        on_off = intentMessage.slots.on_off.first().value
        result_sentence = "Turning lights in the {} {}".format(house_room, on_off)
    else:
        result_sentence = "I did not understand which lights to switch" 
    current_session_id = intentMessage.session_id
    hermes.publish_end_session(current_session_id, result_sentence)
    
if __name__ == "__main__":
    with Hermes("localhost:1883") as h:
        h.subscribe_intent("hugocoolens:switchthelights", subscribe_intent_callback) \
         .start()

thanks in advance
hugo

I think there is a misunderstanding here. My question was not about which command to use to play the sound-file but where to put the command or how to make the above mentioned action file execute it . I’d like to play the sound file after snips has spoken out the result_sentence in the case it understood the intention.

kind regards,
hugo

Hi Hugo,
If you’re asking how you can execute a shell command in Python, that’s with the subprocess.call function. Example:

import subprocess
subprocess.call(["mpg321", "nice_musical_fragment.mp3"])

If you only want to play the mp3 file when your action code has understood you (that is, if there’s a house_room and on_off slot detected), and after Snips has spoken the result sentence, you have to change your code to something like this (I have not tested this):

 def action_wrapper(hermes, intentMessage, conf):
    current_session_id = intentMessage.session_id
    if intentMessage.slots.house_room and intentMessage.slots.on_off:
        house_room = intentMessage.slots.house_room.first().value
        on_off = intentMessage.slots.on_off.first().value
        result_sentence = "Turning lights in the {} {}".format(house_room, on_off)
        hermes.publish_end_session(current_session_id, result_sentence)
        subprocess.call(["mpg321", "nice_musical_fragment.mp3"])
    else:
        result_sentence = "I did not understand which lights to switch" 
        hermes.publish_end_session(current_session_id, result_sentence)

By the way, if you make the house_room and on_off slots required in Snips Console, you don’t need to check whether the slots are available: Snips only calls your intent callback then when all required slots are filled in.

1 Like

You can convert your mp3 file to wav and play it over snips topic hermes/audioServer.
Ex:

 #!/usr/bin/env python2
 # -*- coding: utf-8 -*-
 #
 # Author: Eric Vandecasteele 2018
 # http://blog.onlinux.fr
 #
 # Import required Python libraries
 import paho.mqtt.publish as publish
 SITE = "default"
 MQTT_IP_ADDR = "localhost"
 MQTT_PORT = 1883
 SOUNDFILE = "./ia_ora-notification.wav"
 binaryFile = open(SOUNDFILE, mode='rb')
 wav = bytearray(binaryFile.read())
 publish.single("hermes/audioServer/{}/playBytes/whateverId".format(SITE),
                payload=wav, hostname=MQTT_IP_ADDR)

Thanks for this great reply. I tried it out and it works, apart from one puzzling behaviour: the sound file is played before the resulting sentence is spoken even though in the code the subprocess comes after the command

hermes.publish_end_session(current_session_id, result_sentence)

What could cause this unexpected behavior?

kind regards,
hugo

That’s a threading issue probably. A better solution would be to integrate Eric Vandecasteele’s snippet in your code, after first converting your mp3 file to wav format.

That’s a threading issue probably. A better solution would be to integrate Eric Vandecasteele’s snippet in your code, after first converting your mp3 file to wav format.

I tried Eric Vandecasteele’s snippet in a separate python-script and it works (needed pip install paho-mqtt). Eric’s solution is indeed an answer to my question. However I had as next step in mind to activate some hardware on the rpi (making a gpio pin high or low, addressing something with i2c…), so your solution looked ideal as a starting point for that, I guess it still will be useful, except I don’t have the expected control over what is executed first. I guess there must be a way to achieve that?

kind regards,
Hugo

You can convert your mp3 file to wav and play it over snips topic hermes/audioServer.
Ex:

 #!/usr/bin/env python2
 # -*- coding: utf-8 -*-
 #
 # Author: Eric Vandecasteele 2018
 # http://blog.onlinux.fr
 #
 # Import required Python libraries
 import paho.mqtt.publish as publish
 SITE = "default"
 MQTT_IP_ADDR = "localhost"
 MQTT_PORT = 1883
 SOUNDFILE = "./ia_ora-notification.wav"
 binaryFile = open(SOUNDFILE, mode='rb')
 wav = bytearray(binaryFile.read())
 publish.single("hermes/audioServer/{}/playBytes/whateverId".format(SITE),
                payload=wav, hostname=MQTT_IP_ADDR)

Thanks for the reply, it works after a “pip install paho-mqtt”

kind regards,
hugo

Unfortunately trying to merge Eric Vandecasteele’s
code resulted in neither the result_sentence nor the sound-file being played. This is what I had as code:

#!/usr/bin/env python2
# -*- coding: utf-8 -*-
import paho.mqtt.publish as publish
SITE = "default"
MQTT_IP_ADDR = "localhost"
MQTT_PORT = 1883
SOUNDFILE = "./nice_musical_fragment.wav"
binaryFile = open(SOUNDFILE, mode='rb')
wav = bytearray(binaryFile.read())
import ConfigParser
from hermes_python.hermes import Hermes
from hermes_python.ontology import *
import io

CONFIGURATION_ENCODING_FORMAT = "utf-8"
CONFIG_INI = "config.ini"

class SnipsConfigParser(ConfigParser.SafeConfigParser):
    def to_dict(self):
        return {section : {option_name : option for option_name, option in self.items(section)} for section in self.sections()}


def read_configuration_file(configuration_file):
    try:
        with io.open(configuration_file, encoding=CONFIGURATION_ENCODING_FORMAT) as f:
            conf_parser = SnipsConfigParser()
            conf_parser.readfp(f)
            return conf_parser.to_dict()
    except (IOError, ConfigParser.Error) as e:
        return dict()

def subscribe_intent_callback(hermes, intentMessage):
    conf = read_configuration_file(CONFIG_INI)
    action_wrapper(hermes, intentMessage, conf)


def action_wrapper(hermes, intentMessage, conf):
    current_session_id = intentMessage.session_id
    if intentMessage.slots.house_room and intentMessage.slots.on_off:
        house_room = intentMessage.slots.house_room.first().value
        on_off = intentMessage.slots.on_off.first().value
        result_sentence = "Turning lights in the {} {}".format(house_room, on_off)
        hermes.publish_end_session(current_session_id, result_sentence)
        publish.single("hermes/audioServer/{}/playBytes/whateverId".format(SITE),payload=wav, hostname=MQTT_IP_ADDR)
    else:
        result_sentence = "I did not understand which lights to switch" 
        hermes.publish_end_session(current_session_id, result_sentence)

if __name__ == "__main__":
    with Hermes("localhost:1883") as h:
        h.subscribe_intent("hugocoolens:switchthelights", subscribe_intent_callback) \
         .start()

Does anyone here have an idea how to make this work properly?

kind regards,
hugo

Hi Hugo,

It works fine with one of my action-test program. The only difference is that I publish on hermes/audioServer topic before ending the session.

Hope this will help.

Test code is available here https://github.com/onlinux/snips-zibase/blob/master/test-action-zibase.py

if arg is not None:
for i, item in enumerate(arg):
logger.debug(item)
url = “http://{}/cm?user={}&password={}&cmnd=Power%20on”.format(
item, sonoffUser, sonoffPassword)
logger.debug(url)
try:
resp = requests.get(url)
logger.debug(resp.text)
except requests.ConnectionError, e:
sentence = ‘Désolé mais çà n’a pas marché. Peut être un problème de connexion au sonoff.’
logger.warning(e)

            logger.debug(sentence)
            publish.single("hermes/audioServer/{}/playBytes/whateidver".format(SITE),                              payload=wav, hostname=MQTT_IP_ADDR, client_id="")**
            hermes.publish_end_session(intent_message.session_id, sentence)
2 Likes

That’s a threading issue probably. A better solution would be to integrate Eric Vandecasteele’s snippet in your code, after first converting your mp3 file to wav format.

As I don’t get Erics solution working (I’m a newbie). It would be nice if someone here could modify Koan’s solution such that the threading issue is solved. Koan’s solution is easier to understand for a beginner and it would also be easier to adapt it for driving hardware connected to the raspberry pi (via the respeaker 2 module).

kind regards,
Hugo

Dear Eric,
I tried to make it work with the app " lettherebelight" which is in the Store. Could you have a look at it? I’d appreciate it very much.

kind regards,
Hugo

Hi Hugo,

I tried your assistant and modified the code snippet and it eventually works properly. This is only an example.

Once you have deployed your assistant, create a wav file as /tmp/on_off.wav
and add the line “paho-mqtt” to the requirements.txt file in /var/lib/snips/skills/xxxx.lettherebelight directory, then run setup.sh script in the same directory. This will install the missing module paho-mqtt for snips. You need to do it only once.

I would suggest to work on the full code of your assistant and not the code snippet. The code will look much cleaner.

Here is the snippet modified:

import paho.mqtt.publish as publish
SITE = "default"
SOUNDFILE = "/tmp/on_off.wav"
binaryFile = open(SOUNDFILE, mode='rb')
wav = bytearray(binaryFile.read())
if len(intentMessage.slots.house_room) > 0: #we check if the house_room slot is available
    house_room = intentMessage.slots.house_room.first().value # We extract the value from the slot "house_room"
if len(intentMessage.slots.on_off) > 0: #we check if the on_off slot is available
    on_off = intentMessage.slots.on_off.first().value
#    result_sentence = "Turning lights in the: {}".format(str(house_room+' '+on_off))
#the code line below is better coding according to Koen Vervloesem
    result_sentence = "Turning lights in the {} {}".format(house_room, on_off) # The response that will be said out loud by the TTS engine.
else:
    result_sentence = "I did not understand which lights to switch" 
current_session_id = intentMessage.session_id
publish.single("hermes/audioServer/{}/playBytes/whateverId".format(SITE),payload=wav, hostname="localhost", client_id="")
hermes.publish_end_session(current_session_id, result_sentence)
1 Like

Hey Eric,
You made my day. Thanks a lot for helping me out. It now works like a charm and I learned a lot from your explanation too.

kind regards,
hugo

Hello Eric,
In your code you have chosen:

SOUNDFILE = "/tmp/on_off.wav"

This works, but the problem is that when rebooting the system you have to put back the on_off.wav file in /tmp. I thought I’ll make a directory /home/pi/music and put the on_off.wav file there, change the code to:

SOUNDFILE = "/home/pi/music/on_off.wav"

To my surprise this doesn’t work, could you explain what the reason for this could be?

thanks in advance and kind regards,
hugo

My guess: the user that runs the skill (_snips-skills or something like that) doesn’t have read privileges for /home/pi, but it does for /tmp.

1 Like

My guess: the user that runs the skill ( _snips-skills or something like that) doesn’t have read privileges for /home/pi , but it does for /tmp

Do you know in which log file I could see more or can I put a command in the script to get more debugging information?

kind regards,
hugo

Hi Hugo,

Yes, i would answer the same as Koan. This is a question of read access.
Your script cannot access /home/pi/sounds directory.
I used /tmp just as an example

The best for you, would be to place the wav file under the same directory of your script, which should be
/var/lib/snips/skills/xxxx.lettherebelight

and for instance, define the path as
SOUNDFILE = “./on_off.wav”

Regards

1 Like

A tail -f /var/log/syslog while the intent is activated should give you some information.

1 Like