HOWTO: Convert your Snips skills to Python 3


#1

This little tutorial was inspired by some other forum topics about Python 2 becoming deprecated in less than a year and hence users getting the following error for every app they update:

Error setting up virtualenv, one or more actions might not be able to run. Reason :
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.

I did the following to convert one of my apps to Python 3 and I thought it would be interesting to other app developers:

Step 1: Run 2to3 on your code

Run the command 2to3 *.py on all your Python files. the 2to3 command will give some suggestions for changes in your code. The changes I needed for my What is happening skill were quite limited.

Note that you don’t have to apply all suggestions that 2to3 gives. Some of the suggestions are only needed in specific circumstances. For instance, I didn’t change for key in data.keys(): to the suggested for key in list(data.keys()): in the file snipsTools.py from snips-app-template-py because the code doesn’t use data.keys() as a list, for instance it doesn’t index the list.

Another option is that you use the futurize script, which makes your code compatible with both Python 2 and 3. I’m not interested in bloating my code to support a Python version that becomes deprecated in less than a year, so I didn’t use it.

Also, don’t forget to change the shebang line in the beginning of each Python file to:

#!/usr/bin/env python3

If you’re using the Hermes Python library in your app, add the following in your app’s requirements.txt (because of a bug I found in Hermes Python in step 5):

hermes-python>=0.1.29

Step 2: Create a Python 3 virtual environment

The default setup.sh script from the Snips app template creates a virtual environment for Python 2. Changing this to Python 3 is easy, but I also wanted to handle the situation where my app’s user has his Python 2 version of the app installed and gets a new Python 3 virtual environment after he updates the app, without any remnants of the previous Python 2 environment. So after extending and cleaning up setup.sh, this resulted in the following script:

#/usr/bin/env bash -e

# Copy config.ini.default if config.ini doesn't exist.
if [ ! -e config.ini ]
then
    cp config.ini.default config.ini
fi

PYTHON=`which python3`
VENV=venv

if [ -f "$PYTHON" ]
then

    if [ ! -d $VENV ]
    then
        # Create a virtual environment if it doesn't exist.
        $PYTHON -m venv $VENV
    else
        if [ -e $VENV/bin/python2 ]
        then
            # If a Python2 environment exists, delete it first
            # before creating a new Python 3 virtual environment.
            rm -r $VENV
            $PYTHON -m venv $VENV
        fi
    fi

    # Activate the virtual environment and install requirements.
    . $VENV/bin/activate
    pip3 install -r requirements.txt

else
    >&2 echo "Cannot find Python 3. Please install it."
fi

Step 3: Test your code

Test your changes thoroughly on your Snips machine. Test all actions, especially where you have changed code suggested by 2to3. But 2to3 can’t find all needed changes. For instance, Python 2 and 3 differ a lot in how they handle strings, so that’s an area you should especially pay attention to in your tests (see also step 5).

Step 4: Upload your code

Commit your changes to Git and push them to your GitHub repository. Thanks to the new setup script, there are three situations now when your users update their skills:

  • A new user installs your app for the first time. The script creates a Python 3 virtual environment.
  • The Python 2 app updates to Python 3. The script deletes the Python 2 virtual environment and creates a Python 3 virtual environment.
  • (Later when you keep updating your app) The Python 3 app gets an update. The script already has a Python 3 virtual environment.

In each of these three situations, a Python 3 environment is created, the script activates the virtual environment and installs the requirements. You don’t get the deprecation warning anymore. After this, the Snips skill server runs your actions as Python 3 code, as you can see with ps aux|grep python3.


Snips python 3 instead of python 2
#2

Step 5: Fix things (“Here be dragons”)

As expected after such a big switch as from Python 2 to 3, some things will break. With two of my apps, I ran into the same issue with time interval values in Hermes Python. As @Anthony commented on GitHub, this will be fixed in Hermes Python 0.1.29.
Maybe some other issues will occur. If you’re migrating your apps to Python 3 and you see some weird error messages coming from Snips internal components, open an issue on GitHub like I did and tell the Snips developers what’s going on.


#3

The issue is now fixed in version 0.1.29 that was released today ! https://pypi.org/project/hermes-python/0.1.29/


#4

Thanks for the fast fix, Anthony! I added some information about using hermes-python>=0.1.29 in requirements.txt in my original post.


#5

Great ! Does this release fix your issues ?


#6

It’s working perfectly now!


#7

There is an exclamation mark missing in the first line:

This is right:
#!/usr/bin/env bash -e


#8

Good catch! I just copied the setup.sh file from the Snips app template and adapted it, and I didn’t notice it. So it should be fixed there too.


#9

I did put this in my actions.
All my actions works with python 2.7 or 3 since start as my testing environement is python 3

Anyway after sam update-assistant I still have exact same issue

ERROR:snips_template                    : Permission denied (os error 13)
Setting up Python actions
Running setup.sh & generating virtual environment for kiboost.DateAgeManips
âś– Error setting up virtualenv, one or more actions might not be able to run. Reason :
You are using pip version 18.1, however version 19.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

Pip is 19.0.1

Don’t know what to do to get my actions working back


#10

my snippets are still download in

/usr/share/snips/assistant/snippets/kiboost.myactionname/python2/kiboost

 python3 --version
 Python 3.5.3

 pip --version
 pip 19.0.1 from /usr/local/lib/python2.7/dist-packages/pip (python 2.7)

#11

This tutorial only works for complete action code, e.g. from GitHub, not for snippets you add in the Snips Console. As @Anthony told in another post, Python 3 isn’t supported there yet.


#12

Ok thanks.

I deleted snippets and skills folders, re updated assistant, and did snips-template render, not tested all actions but a few and it works.


#13

Hi, I successfully converted snips skill to python 3. My python file uses this line of code

intent=[..]
h.publish_start_session_action("default","",intent,True,"" )

With new version of hermes I noticed, looking at python file, that that function changed from

publish_start_session_action(self, site_id, session_init_text, session_init_intent_filter, session_init_can_be_enqueued, custom_data)

to

publish_start_session_action(self, site_id, session_init_text, session_init_intent_filter, session_init_can_be_enqueued, session_init_send_intent_not_recognized, custom_data)

there is this new variable

session_init_send_intent_not_recognized

that is not documented inside the file hermes.py
So I suppose from variable name that it should be a boolean that indicates whether to initialize session base on the intent if it has been recognize or not?


#14

Apperently this has been changed in commit 704199b8e5597eb22ac27dca87c12ab056d79501. I don’t know what the new parameter does.


#15

Hi, I just put it to False and my old intents seems to work now.
Is there a sam command to see the list of intents installed on my raspberry?
Because I added a new intent from website, did a “sam update-assistant” and I didn’t have any error. But when I try the intent on my raspberry it seems that the new intent is not installed at all.


#16
cat /usr/share/snips/assistant/assistant.json | python3 -c 'import sys, json; [print(intent["name"]) for intent in json.load(sys.stdin)["intents"]]'

#17

Hmm apparently it has been installed but is not being recognized. It always recognize another intent which is similar (“play [song_name]” and “play my favorite song”, the second one is the new intent I added) even though from snips website the 2 intents are recognized without any problem.
Anyway thanks.