Receive DTMF and Speech Input

Overview

You can use speech input or dual-tone multi-frequency (DTMF) tones (a.k.a. Touch-Tone) to route callers or otherwise change call flows for applications such as interactive voice response (IVR), virtual assistants, and mobile surveys.

Prerequisites

To get started, you need a Plivo account — sign up with your work email address if you don’t have one already. You must have a voice-enabled Plivo phone number to receive incoming calls; you can rent numbers from the Numbers page of the Plivo console, or by using the Numbers API. If this is your first time using Plivo APIs, follow our instructions to set up a .NET development environment and a web server and safely expose that server to the internet.

Detect DTMF input

How it works

Receive DTMF

This example shows a multilevel IVR phone application that uses digit press input captured using the GetInput XML element. A virtual assistant answers incoming calls and offers the caller three choices: “Press 1 for your account balance. Press 2 for your account status. Press 3 to speak to a representative.” If the caller enters 1 or 2, the application will retrieve the requested information and play the caller a text-to-speech message. If the caller presses 3, the application will redirect the caller to the second branch, which offers two new choices: “Press 1 for sales. Press 2 for support.” The application then connects the caller with the requested department.

Create an MVC controller to detect DTMF input

In Visual Studio, create a controller named MultilevelIvrController.cs and paste into it this code.

using System;
using System.Collections.Generic;
using Plivo.XML;
using System.Diagnostics;
using Microsoft.AspNetCore.Mvc;

namespace Receivecall.Controllers {
  public class MultilevelIvrController: Controller {

    //  Welcome message, first branch
    String WelcomeMessage = "Welcome to the demo. Press 1 for your account balance. Press 2 for your account status. Press 3 to speak to a representative";
    // Message for second branch
    String RepresentativeBranch = "Press 1 for sales. Press 2 for support";
    // Message that Plivo reads when the caller does nothing
    String NoInput = "Sorry, I didn't catch that. Please hang up and try again";
    // Message that Plivo reads when the caller presses a wrong digit
    String WrongInput = "Sorry, that's not a valid input";
    
    // GET: /<controller>/
    public IActionResult Index() {
      var resp = new Response();
      GetInput get_input = new GetInput("", new Dictionary < string, string > () {
        {
          "action",
          "https://<ngrok_identifier>.ngrok.io/multilevelivr/firstbranch/"
        },
        {
          "method",
          "POST"
        },
        {
          "digitEndTimeout",
          "5"
        },
        {
          "inputType",
          "dtmf"
        },
        {
          "redirect",
          "true"
        },
      });
      resp.Add(get_input);
      get_input.AddSpeak(WelcomeMessage, new Dictionary < string, string > () {});
      resp.AddSpeak(NoInput, new Dictionary < string, string > () {});
    
      var output = resp.ToString();
      return this.Content(output, "text/xml");
    }
    public IActionResult FirstBranch() {
      String digit = Request.Form["Digits"];
      Debug.WriteLine("Digit pressed : {0}" + digit);
    
      var resp = new Response();
    
      if (digit == "1") {
        // Add Speak XML element
        resp.AddSpeak("Your account balance is $20", new Dictionary < string, string > () {});
      }
      else if (digit == "2") {
        // Add Speak XML element
        resp.AddSpeak("Your account status is active", new Dictionary < string, string > () {});
      }
      else if (digit == "3") {
        String getinput_action_url = "https://<ngrok_identifier>.ngrok.io/multilevelivr/secondbranch/";
    
        // Add GetInput XML element
        GetInput get_input = new GetInput("", new Dictionary < string, string > () {
          {
            "action",
            getinput_action_url
          },
          {
            "method",
            "POST"
          },
          {
            "digitEndTimeout",
            "5"
          },
          {
            "inputType",
            "dtmf"
          },
          {
            "redirect",
            "true"
          },
        });
        resp.Add(get_input);
        get_input.AddSpeak(RepresentativeBranch, new Dictionary < string, string > () {});
        resp.AddSpeak(NoInput, new Dictionary < string, string > () {});
      }
      else {
        // Add Speak XML element
        resp.AddSpeak(WrongInput, new Dictionary < string, string > () {});
      }
    
      Debug.WriteLine(resp.ToString());
    
      var output = resp.ToString();
      return this.Content(output, "text/xml");
    }
    // Second branch of IVR phone tree
    public IActionResult SecondBranch() {
      String FromNumber = Request.Form["From"];
      var resp = new Response();
      String digit = Request.Form["Digits"];
      Debug.WriteLine("Digit pressed : {0}" + digit);
    
      // Add Speak XMLTag
      if (digit == "1") {
        Dial dial = new Dial(new
        Dictionary < string, string > () {
          {
            "callerId",
            FromNumber
          },
          {
            "action",
            "https://<ngrok_identifier>.ngrok.io/multilevelivr/action/"
          },
          {
            "method",
            "POST"
          },
          {
            "redirect",
            "false"
          }
        });
    
        dial.AddNumber("<number_1>", new Dictionary < string, string > () {});
        resp.Add(dial);
      }
      else if (digit == "2") {
        Dial dial = new Dial(new
        Dictionary < string, string > () {
          {
            "callerId",
            FromNumber
          },
          {
            "action",
            "https://<ngrok_identifier>.ngrok.io/multilevelivr/action/"
          },
          {
            "method",
            "POST"
          },
          {
            "redirect",
            "false"
          }
        });
    
        dial.AddNumber("<number_2>", new Dictionary < string, string > () {});
        resp.Add(dial);
      }
      else {
        resp.AddSpeak(WrongInput, new Dictionary < string, string > () {});
      }
    
      Debug.WriteLine(resp.ToString());
    
      var output = resp.ToString();
      return this.Content(output, "text/xml");
    }
  }
}

Run the project and you should see your application in action at http://localhost:5000/multilevelivr/.

Control the gathering of DTMF input

You can improve DTMF collection by using attributes available for the GetInput XML element, such as digitEndTimeout, numDigit, finishOnKey, and executionTimeout.

digitEndTimeout sets the maximum time interval between successive digit inputs. The default value is auto and other allowed values are 2 to 10 seconds. If the user provides no new digits within the digitEndTimeout period, the digits entered to that point will be processed.

numDigits sets the maximum number of digits the user can provide on the current call. The default value is 32 and the allowed values are 1 to 32.

If the user provides more digits than the value of numDigits, Plivo will send only the number of digits specified as numDigits to the action URL; additional digit inputs will be ignored. For example, if numDigits is specified as “4” and the user enters five digits, the last digit will be ignored.

finishOnKey defines a key that users can press to submit the digits they entered. The default value is # and additional allowed values are 0-9, *, <empty string>, and ”none.” When you set the value to <empty string> or “none,” DTMF input collection ends depending on the digitEndTimeout or the numDigits attribute.

Note: These three attributes apply to input types dtmf and dtmf speech and do not apply to input type speech. If all three of these attributes are specified, the priority is for finishOnKey.

executionTimeout sets the maximum time during which Plivo detects input. You can use this timeout to tell the application to process the next element in the XML response when a user doesn‘t provide input during the call. The default value is 15 seconds, and allowed values are 5 to 60 seconds.

Detect speech input

The GetInput XML element can also capture speech input.

How it works

Receive speech

This example shows how to implement a simple IVR phone tree. A virtual assistant answers the call and offers the caller two choices: “Say sales to talk to a sales representative. Say support to talk to a support representative.”

If the caller says “sales,” the caller will be connected to a sales representative; if the caller says “support,” they will be connected to a support representative.

Create an MVC controller to detect speech input

In Visual Studio, create a controller named IvrspeechController.cs and paste into it this code.

using System;
using System.Collections.Generic;
using Plivo.XML;
using System.Diagnostics;
using Microsoft.AspNetCore.Mvc;

namespace Receivecall.Controllers {
  public class IvrspeechController: Controller {
    //  Welcome message, first branch
    String WelcomeMessage = "Welcome to the demo. Say sales to talk to a sales representative. Say support to talk to a support representative";
    // Message that Plivo reads when the caller does nothing
    String NoInput = "Sorry, I didn't catch that. Please hang up and try again";
    // Message that Plivo reads when the caller speaks something unrecognized
    String WrongInput = "Sorry, that's not a valid input";

    public IActionResult Index() {
      var resp = new Response();
      GetInput get_input = new GetInput("", new Dictionary < string, string > () {
        {
          "action",
          "https://<ngrok_identifier>.ngrok.io/ivrspeech/firstbranch/"
        },
        {
          "method",
          "POST"
        },
        {
          "interimSpeechResultsCallback",
          "https://<ngrok_identifier>.ngrok.io/ivrspeech/firstbranch/"
        },
        {
          "interimSpeechResultsCallbackMethod",
          "POST"
        },
        {
          "inputType",
          "speech"
        },
        {
          "redirect",
          "true"
        },
      });
      resp.Add(get_input);
      get_input.AddSpeak(WelcomeMessage, new Dictionary < string, string > () {});
      resp.AddSpeak(NoInput, new Dictionary < string, string > () {});
    
      var output = resp.ToString();
      return this.Content(output, "text/xml");
    }
    // IVR phone tree
    public IActionResult FirstBranch() {
      String speech = Request.Form["Speech"];
      String FromNumber = Request.Form["From"];
      Debug.WriteLine("Speech Input is :" + speech);
      Dial dial = new Dial(new
      Dictionary < string, string > () {
        {
          "callerId",
          FromNumber
        }
      });
    
      var resp = new Response();
    
      if (speech == "sales") {
        dial.AddNumber("<number_1>", new Dictionary < string, string > () {});
        resp.Add(dial);
      }
      else if (speech == "support") {
        dial.AddNumber("<number_2>", new Dictionary < string, string > () {});
        resp.Add(dial);
      }
      else {
        // Add Speak XML element
        resp.AddSpeak(WrongInput, new Dictionary < string, string > () {});
      }
    
      Debug.WriteLine(resp.ToString());
    
      var output = resp.ToString();
      return this.Content(output, "text/xml");
    }
  }
}

Save the controller and run it and you should see your application in action at http://localhost:5000/ivrspeech/.

Speech recognition attributes

Speech models

Different applications may benefit from different automatic speech recognition (ASR) models, which you can specify using the the GetInput XML element‘s speechModel attribute. By default, it has a value of default, which is suitable for long-form audio, such as dictation, but you can also try command_and_search for shorter audio clips, such as when you expect callers to use voice commands or voice search, or phone_call, if you want to transcribe audio from a phone call. Explore the models and see which works best for your use case.

Example XML:

<Response>
<GetInput action="https://<yourdomain>.com/action/" method="POST" inputType="speech" speechModel="command_and_search" redirect="true">
<Speak>Welcome to the demo. Say sales to talk to a sales representative. Say support to talk to a support representative</Speak>
</GetInput>
<Speak>Sorry, I didn't catch that. Please hang up and try again later.</Speak>
</Response>

Hints

You can use the hints attribute to potentially improve speech transcription results by defining words and phrases that are common in your use case. For example, a call center where callers use voice commands to connect to various departments can use the names of the departments as hints.

Allowed values: a non-empty string of comma-separated phrases
Limitations are:
- Phrases per request: 500
- Characters per phrase: 100
- Characters per request: 10,000

Example XML:

<Response>
<GetInput action="https://<yourdomain>.com/action/" method="POST" inputType="speech" hints="sales,support" redirect="true">
<Speak>Welcome to the demo. Say sales to talk to a sales representative. Say support to talk to a support representative</Speak>
</GetInput>
<Speak>Sorry, I didn't catch that. Please hang up and try again later.</Speak>
</Response>

Controlling the gathering of speech input

You can improve the functionality of speech input collection by using GetInput XML attributes such as speechEndTimeout, language, profanityFilter, and executionTimeout.

speechEndTimeout sets the time that Plivo waits for more speech input after silence is detected. The default value is auto; other allowed values are 2 to 10 seconds. If the user doesn‘t provide new speech input within the speechEndTimeout period, the speech collected to that point will be processed.

language specifies the language and national/regional dialect of the audio to be recognized on calls. The default language for speech detection is en-US. You can choose your preferred language from the list of supported languages.

profanityFilter: If a user speaks any profane words, Plivo can filter them out during transcription if you set this attribute to true. The profanity filter applies only to single words — it doesn‘t work for a combination of words. The default value is false.

Note: These three attributes apply to input types speech and dtmf speech and do not apply to input type dtmf.

Example XML:

<Response>
<GetInput action="https://<yourdomain>.com/action/" method="POST" inputType="speech" speechEndTimeout="5" language="en-IN" profanityFilter="true" executionTimeout="25" redirect="true">
<Speak>Welcome to the demo. Say sales to talk to a sales representative. Say support to talk to a support representative</Speak>
</GetInput>
<Speak>Sorry, I didn't catch that. Please hang up and try again later.</Speak>
</Response>

Real-time speech recognition

You can use the interimSpeechResultsCallback attribute to perform real-time speech recognition. If you specify a URL for your application server to this attribute, you can receive real-time callbacks of the user’s recognized speech while the user is still speaking on the call. Plivo sends the transcribed result to your server URL with attributes such as UnstableSpeech, Stability, StableSpeech, and SequenceNumber.

UnstableSpeech holds the interim transcribed result of the user’s speech, which may be refined when more speech is collected from the user.
Stability is an estimate of the likelihood that the recognizer will not change its guess about the interim UnstableSpeech result. Values range from 0.0 (completely unstable) to 1.0 (completely stable).
StableSpeech holds the stable transcribed result of the user’s speech.
SequenceNumber holds the sequence number of the interim speech callback, which helps you order incoming callback requests.

Example XML:

<Response>
<GetInput action="https://<yourdomain>.com/action/" method="POST" interimSpeechResultsCallback="https://<yourdomain>.com/interimcallback/" interimSpeechResultsCallbackMethod="POST" inputType="speech" redirect="true">
<Speak>Welcome to the demo. Say sales to talk to a sales representative. Say support to talk to a support representative</Speak>
</GetInput>
<Speak>Sorry, I didn't catch that. Please hang up and try again later.</Speak>
</Response>

Data logging preferences

You can use the GetInput XML element’s log attribute to manage input logging preferences. It defaults to true, but if you define it to false, logging will be disabled and Plivo will not log digit and speech input.

Essential Guides

Advanced Guides

Audio Streaming

Receive DTMF and Speech Input

Overview

Prerequisites

Detect DTMF input

How it works

Create an MVC controller to detect DTMF input

Control the gathering of DTMF input

Detect speech input

How it works

Create an MVC controller to detect speech input

Speech recognition attributes

Speech models

Hints

Controlling the gathering of speech input

Real-time speech recognition

Data logging preferences

Essential Guides

Advanced Guides

Audio Streaming

​Overview

​Prerequisites

​Detect DTMF input

​How it works

​Create an MVC controller to detect DTMF input

​Control the gathering of DTMF input

​Detect speech input

​How it works

​Create an MVC controller to detect speech input

​Speech recognition attributes

​Speech models

​Hints

​Controlling the gathering of speech input

​Real-time speech recognition

​Data logging preferences

Overview

Prerequisites

Detect DTMF input

How it works

Create an MVC controller to detect DTMF input

Control the gathering of DTMF input

Detect speech input

How it works

Create an MVC controller to detect speech input

Speech recognition attributes

Speech models

Hints

Controlling the gathering of speech input

Real-time speech recognition

Data logging preferences