Batch testing assistant


#1

Hi,

Actually we have in console a way to test a sentence. Nice.

Now, having different apps with lot of intents and tons of possible sentences, when modifying one intent, or adding another app, all can be mess, or just a few sentences. For example, I ever had a few sentences that triggered a wrong intent, so I modified them, it works but then, other sentence became messing intents.

So now, I keep a set of sentence that I always try once I’m ready to update the assistant on the pi, to be sure that I didn’t mess other things.

And here we are, testing around 30 sentences at the end to be sure all is working fine is … a pain !

So, could we have a page dedicated to batch testing, where we could import a set of sentences, even store it for later test, and test all sentences in a batch ?
Even better could be a table of:

this is my sentence ------ here is the intent it should trigger ----- here intent triggered after test (green/red)

Sincerely, this would save a lot of time, and provide an efficient global assistant testing.

Please … :yum:

PS: you note I didn’t asked to specify which slots should be defined with the intent… I keep for v2 :sweat_smile:


#2

Hey, sorry for late reply. We have been experimenting with this kind of functionality, hopefully we can offer something in the UI in the future. I agree that it would be super useful.


#3

Thanks, this is on top of my priority list !


#4

Any news on this subject ? my fingers are numb with all copy/paste :rofl:


#5

Hehe I understand. It’s still something we’re looking at, but it’s not something that we will be offer in the very short term - it will take longer.


#6

ok, I just write a php script to do it on my own.
Now can tests several sentences at once with intent and slots results.

:relieved:


#7

Go for it. :+1: I’ll post here when I have any updates about plans to integrate this in the console.


#8

I have it working, just coloring red/green with expected intent is match or not.
May publish it on github if someone is interested.
It is a php script so it needs apache server


#9

Ok, done it with python as for lot of sentences, php will timeout.
php had the advantage of text coloring in result, but not really the right tool for that.
I made ad html result later with better formating and colors.

I hope not infringing SNIPS policy by connecting to console from python script :relieved:

:stuck_out_tongue_winking_eye:


#10

Ahah !!

matched: 46 | unmatched: 0 | total: 46

So much easier to fine tune assistant with batch testing !!! :sweat_smile: