Voice commands (speech recognition) and speech Synthesis with Electron Framework

Electron Framework uses the embedded chromium framework, therefore most (not all) of  the features availables on Google Chrome, are available for you in an Electron App.

In this article, you'll learn how to add voice commands and synthesize text using Artyom.js. Artyom.js is an useful wrapper of Speech Recognition and Speech Synthesis, which allow to add voice commands and speech text easily with a couple of lines of code.

Include artyom.js

Add artyom into your project executing the following command in the node.js command line :

npm install artyom.js --save

Is recommendable to read the documentation of artyom here or visit the official repository here.

Include the artyom.js file in your project using a script tag in the head of your html document.

<script src="path-to-file/artyom.min.js"></script>

Speech synthesis

Pitifully, with Electron Framework you'll be not able to choose the voice that you want, instead the default voice of the computer will be selected. That means, that you are limited to your default language i.e if your computer is in spanish, you'll be able only to read spanish fluid. Unless you want to write your own speech synthesis module reading the documentation of the speech Synthesis API and learn how  to choose a voice, which becomes useless as it may be only available for the computer in which you are working (not maintainable).

To synthesize text, use the artyom.say function :

artyom.say("I will read this text on your computer, great isn't?");

Note: Artyom solves in Google Chrome the limitation of 200 characters, however in an Electron App, this limit doesn't exist as no remote service is used. The speech synthesis uses the OS local voice.

Voice commands

To add voice commands to our Electron App we'll use the artyom.addCommands function. Every command is a literal object with the words that trigger the command in an array and an action parameter which is a function that will be triggered when the voice matches with the command.

The following command answers hello if the user says hello or hi:

var adminName = "Carlos";

artyom.addCommands({
    indexes: ["Hello","Hi"],
    action: function(){
        artyom.say("Hello, how are you today "+ adminName +"?");
    }
});

Now that you know how to add commands, let's get started with a basic example.

Example

Add 3 buttons and attach every of the following functions to them on the onclick event.

One will start artyom in continuous mode and other in one command listener.

Note: In Google Chrome a https connection is required to use the continuous mode, however in the Electron App it works without any kind of inconvenient.

<button type="button" onclick="StartArtyomOneCommand();">Start artyom one command</button>
<button type="button" onclick="StartArtyomContinuous();">Start artyom continuous assistant</button>
<br>
<button type="button" onclick="StopArtyom();">Stop recognition</button>

<script>
function StartArtyomOneCommand(){
    console.log("One command");
    if(artyom.isRecognizing()){
        return alert("Stop artyom first !");
    }

    //Although the voice can't be changed,
    // You need to set the language for the speech
    // Recognition, see the documentation for more examples
    return artyom.initialize({
        lang:"en-GB",
        debug:true,
        continuous:false,
        listen:true
    });
}

function StartArtyomContinuous(){
    console.log("Continuous commands");
    if(artyom.isRecognizing()){
        return alert("Stop artyom first !");
    }

    // You can create a permanent voice assistant
    // if you want using the continuous mode !
    return artyom.initialize({
        lang:"en-GB",
        debug:true,
        continuous:false,
        listen:true
    });
}

function StopArtyom(){
    artyom.fatality();
}
</script>

Now add some commands and add the error listener for artyom on the window.onload event :

<script>
'use strict';
    window.onload = function(){

        // Add the error listeners
        artyom.when("ERROR",function(err){
            console.error("An error ocurred : ", err.code);
        });

        console.log("Artyom is ready");
        
        // Important, add the commands to process.
        artyom.addCommands([
            {
                indexes:["Hello","Hi"],
                action: function(i){
                    artyom.say("Hello, how are you today?");
                }
            },
            {
                indexes:["Say * please"],
                smart:true,
                action: function(i,wildcard,sentence){
                    artyom.say(wildcard);
                }
            },
            {
                indexes:["Text content *"],
                smart:true,
                action: function(i,wildcard,sentence){
                    document.getElementById("text-content").value = wildcard;
                }
            },
            {
                indexes:["write * in the console"],
                smart:true,
                action: function(i,wildcard,sentence){
                    console.log(wildcard);
                }
            }
        ]);
    };
</script>

Finally , just build your app using npm start and test the commands that we've just added. The speech Synthesis doesn't require demo as you just need to pass a string as first parameter to the artyom.say function.

You can see the previous example in the official our code world electron examples in the repository here.