Is it possible to modify the default timeout on no response from the user

I have noticed that the session times itself out when we do not give input to the ASR after 4 seconds, is it possible to change that value programmatically in android?

Hi @manimaran

This is not something that is supported officially, you can however tweak the asr endpointing rules to augment that time, I’ll let @Garvys explain

Hello @manimaran !
We implemented a few rules so the ASR can stop listening by automatically detecting the end of a sentence (called endpointing rules). You can modify them but it is kinda hacky :frowning:
The idea is that:

  • if the ASR thinks he understood a full sentence, it will stop listening after 0.5 s
  • if the ASR thinks he understood a part of a sentence, it will stop listening after 1.0 s

They can be found in custom_asr/config.json and it looks something like that :

    "endpointingRules": {
        "rule1": {
            "mustContainNonsilence": False,
            "minTrailingSilence": 4.0,
            "maxRelativeCost": 1e10,
            "minUtteranceLength": 0.0
        },
        "rule2": {
            "mustContainNonsilence": True,
            "minTrailingSilence": 0.5,
            "maxRelativeCost": 3.5,
            "minUtteranceLength": 0.0
        },
        "rule3": {
            "mustContainNonsilence": True,
            "minTrailingSilence": 1.0,
            "maxRelativeCost": 8.0,
            "minUtteranceLength": 0.0
        },
        "rule4": {
            "mustContainNonsilence": True,
            "minTrailingSilence": 2.0,
            "maxRelativeCost": 1e10,
            "minUtteranceLength": 0.0
        },
        "rule5": {
            "mustContainNonsilence": False,
            "minTrailingSilence": 0.0,
            "maxRelativeCost": 1e10,
            "minUtteranceLength": 10.0
        }
    }

I believe you are interested in the first rule that is cutting the ASR if after 4 seconds only silence has been detected.
Once the rules values are modified, you have to restart snips-asr in order for the changes to be taken into account.
Be aware that any modifications on other keys of the json can potentially break the model. Also, if you are using snips-injection, you should have multiple custom_asr folder. You should then modify the correct one :wink:

3 Likes

Thanks for the tip :kissing_heart:

Hello, I am modifying Snips config file to make it more patient for those who speak slowly between the words.

I would like to know what’s the function of rule 1 , 2 , 3 , 4 , 5 respectively as well as these parameters.
mustContainNonsilence”: False,
minTrailingSilence”: 4.0,
maxRelativeCost”: 1e10,
minUtteranceLength”: 0.0

Thanks!

Hey !

   The endpointing rule is a disjunction of conjunctions.  The way we have
   it configured, it's an OR of five rules, and each rule has the following form:
      (<contains-nonsilence> || !rule.must_contain_nonsilence) &&
       <length-of-trailing-silence> >= rule.min_trailing_silence &&
       <relative-cost> <= rule.max_relative_cost &&
       <utterance-length> >= rule.min_utterance_length)

More info in the source code here : https://github.com/kaldi-asr/kaldi/blob/master/src/online2/online-endpoint.h
Cheers