Lesson 2: Compose email by voice

Don't type your emails -- speak them!

Goal

Create an web page to compose emails by voice.

Do it now

If you're happy with codelab1.html and got everything working, continue working in that file. If you'd like to start fresh, you can instead use codelab1solution.html to begin this lesson.

2.1 Create a space to hold the recognition text.

Since we'll be composing an email, we need a bigger place to display the text, so let's replace that little input field with a big box for text. This also allows us to add formating styles to the text, so let's provide a place to display the final text in black, and interim text in grey.

The speech recognizer returns interim results quickly, but they may change until the speech recognizer marks them as final. Up until now, we have only displayed final results. By displaying the interim text in grey, we provide swift feedback to the user.

Replace the <form> through </form> element with the following HTML. This creates two <span> elements, one that displays black text and one that display grey text.

<div id="results">
  <span id="final_span" style="color:black"></span>
  <span id="interim_span" style="color:gray"></span>
</div>

Also, replace your code that sets "reco.finalResults(..)" with the following two lines that refer to these new <span> elements.

reco.finalResults('final_span');
reco.interimResults('interim_span');

Finally, add a CSS style that draws a border box around the results <div> and makes the font and layout look nice. Put this just above the <h1> element.

<style>
  #results {
    font-size: 20px;
    font-weight: bold;
    border: 1px solid #ddd;
    padding: 15px;
    text-align: left;
    vertical-align: text-top;
    min-height: 150px;
    width: 80%;
    display: inline-block;
  }
</style>

Test it. Refresh the page and click on the microphone. As you speak, you'll see grey interim text flow out quickly, and then being replaced by black text as the recognizer makes a final determination of the results.

2.2 Use continuous recognition.

So far, recognition stops as soon as you pause speaking. Great for search queries, but not so good for composing an email unless you can say everything in one breath.

Add the following code underneath the "reco.interimResults(..)" line to tell the recognizer to listen continuously until you click the microphone again to stop.

reco.continuous = true;

Test it. Refresh the page and click on the microphone and speak and speak and speak and pause and speak some more. Click the microphone again when your done.

2.3 Send the email.

Add a "Create Email" button. Put this HTML after the existing button.

<button onclick="emailButton()">Create Email</button>

Now add the code that executes when the "Create Email" button is clicked. We'll also change how we handle onEnd when speech recognition completes so that it creates the email from the final_transcript. Replace your reco.onEnd callback code with the following code.

var create_email_on_end = false;

// Handler when speech recognition completes.
reco.onEnd = function() {
  if (create_email_on_end) {
    create_email_on_end = false;
    createEmail(reco.final_transcript);
  }
};

// Handler when user clicks "Create Email" button.
function emailButton() {
  if (reco.inProgress()) {
    // Wait for recognition to end before calling createEmail().
    create_email_on_end = true;
    reco.stop();
  } else {
    // Recognition has already ended, call createEmail() now.
    createEmail(reco.final_transcript);
  }
  reco.onState('complete');
}

// Create email by splitting string s into subject and body.
function createEmail(s) {
  // Determine a good place to split it: end of first line, else at a space.
  var n = s.indexOf('\n');
  if (n < 0 || n >= 80) {
    n = 40 + s.substring(40).indexOf(' ');
  }
  var subject = encodeURI(s.substring(0, n));
  var body = encodeURI(s.substring(n + 1));
  // Open default email provider.
  window.location.href = 'mailto:?subject=' + subject + '&body=' + body;
}

Test it. Refresh the page and click on the microphone. When you're done, click "Create Email".

If it doesn't work, or if you want to change your default email provider, see chrome://settings/handlers

Next up: Lesson 3


This Codelab by Glen Shires. Visit me on Google+!