Speech Recognition cased to work in 0.63.3

Hi,

Currently I’m evaluating the ability of Snips to run on a Android Watch. For now, we are… blessed with the TicWatch, but apart from that, following the instructions in your docs section for android left me with an testapp, that actually works on the watch! Thank you for creating this.

But I’m here because I already run into some problems, of which the one described in the title is the most strange one.

In that docs section, one hint tells me

This snippet is provided as an example, check the release notes page to check what’s the latest version available.

which is true, because in the docs section this one here is offered for usage

dependencies {
// ...
implementation('ai.snips:snips-platform-android:0.62.3@aar') {
    transitive = true
}
}

But the latest version is supposed to be the 0.63.3, as far as I can tell.
So i changed that value inside the build.gradle file and… guess what? The speech recognition cased to work.

To be more precise, the “hotword” (or is it “wakeword”?) recognition seems to refuse to acccept my call for “hey snips”. It justs hangs there and… nothing happens. I tied the “setOnPlatformReady” (and others) Callback to an UI-textview,so I can tell whats happening inside the program actually.
And so I can see that it waits for the “hotword”, but never reacts to it, as it does with version 0.62.2.
The log-output seems to be mostly equal. I will extract and attach the logcat if its necessary.

The zip-file in usage is the same, but it was created yesterday. I know, it says to update your zip-file, but its version didn’t change, so I cannot see why it should need a new one?

So, anyone can tell me what its problem might be?

So, am I the only one who have this problem?
I noticed that snips wont terminate in a clean manner after calling client.disconnect(), so some Thread-3 keeps the android app from closing. Hoping that the latest version may work better, I came back here … and yes, remembered about this error report. On which no one did reply to :frowning:

Hi @imunixx!

Sorry I didn’t see you post earlier as is wasn’t in the android category…

Could you provide logs of the you.app.id:snipsProcessingService process after enabling logs on the snips platform client ?

Hello,
pardon me for not putting this inside the android section.

I… think you dont want me to paste the whole log here inside this reply. The attachment button is for pictures only, so I’ll get this to you using pastbin: https://pastebin.com/hc8BKnZZ

To be honest, the only piece of suspicious log seems to be the second last line, which tells about a buffer that cannot be send to.
What you get there to see is all that happens after the assistant.zip has been unpacked (or the existing one was reused), the snips Client had been created and started (client.connect(context)). Also the “OnPlatformReady”-Implementation has been called and even the “Say the hotword, ask your query and let the magic happen” Text did appear in the apps log (its the latest entry right now).

If I call “Hey Snips”… just nothing happens.

Hi @imunixx!

You are using the OpenSLES sound impl (the default one that is broken in the current release a will be fixed in the next one) make sure to switch to a streaming version. A simple example (no really robust/prod ready) here:

The important parts are enabling the streaming and running the capture

A more robust impl (in kotlin) can be found in this other demo application (it still uses an older version of the platform but this part didn’t change)

Dear fredszaq,

thank you for your response and please pardon my delayed reply. I was pulled away from that project for some time and came back just today.
Unfortunally snips 0.64 did get productive during that time, and of course I tried if somethings did change from 0.63.3… and it did.
Now snips just triggers the “onPlatformError” with “RemoteService died” (that google tts thingy) as message… which prevents me from even remotely trying out what you did suggest. I found another thread about that kind of error, but not in the android section.

Should I wait for the next version? Thing is, another word needs to be available as intent, and training it now would yield something that isn’t compatible with 0.62.3 any more, isnt’ it?

try to get the logs of the you.app.id:snipsProcessingService process after enabling the logs on the client. This should give us some insight

Hi fredszaq.

logs from logcat: https://pastebin.com/EE0f1dSx

This is happened if we are using a today trained model with the current version 0.64.0.
We trained a model on last friday, 27th september, and this model works with the platform version 0.62.3 perfectly.

If we are using the pretrained model from today with the version 0.62.3, the wakeword was recognized, but the speech is not. The session endet with: read: unexpected EOF!

The Client was instancieted with:
final SnipsPlatformClient client = new SnipsPlatformClient.Builder(assistantDir)
.enableDialogue(true)
.enableHotword(true)
.enableSnipsWatchHtml(false)
.enableLogs(true)
.withHotwordSensitivity(0.5f)
.enableStreaming(false)
.enableInjection(false)
.build();

While “assistantDir” is a the asset folder with the underlaying assistant.zip.

Thanks.

Hi fredszaq.

After a little bit of validating this bug, I properbly found a solution.
After I inserted:

packagingOptions {
        exclude 'lib/armeabi/'
        exclude 'lib/mips/'
        exclude 'lib/mips64/*'
    }

to the gradle build, the today pretrained model with the platform 0.64.0 is working.
Is this the current issue that we have?

Thanks

yes these lines are needed for the platform to work properly. they seem to have been removed from the getting started doc from some reason, I’ll see to add them back (if you do not exclude the armeabi package, armv7 devices will see the jna lib from this folder and try to load the platform from there as well, but the platform is only compatible with armv7 and hence should be loaded from the armeabiv7 folder… android is a little dumb on this one

Okay, thanks for sharing and have a nice day.