Metrics on accuracy

Does Snips maintain any metrics on intent prediction accuracy?

I’ve got training data for about 45 intents. I’d like to run through a battery of phrase classifications and get an average for the confidence for each intent. Then I could concentrate on increasing the accuracy of just the low scoring intents. Does such an animal exist?

If not, I will write a test frame that will take a phrase and the intent that should be returned. It’ll spit out the resulting average for each intent, a list of errors, and whatever I can think of that might also be useful.

Hi @Shotgun167, I think that the unit tests in the console should do the job.

That is a lot of what I’m aiming for, but it doesn’t give me a concise report against all of my intents, and it requires the use of the Console. I have a need for a tool that I can run as part of my team’s CI.