How to create your own voice assistant in ReactJS using Artyom.js

AI is in even on your smart phone, it's there every time you ask some stupid question to your iPhone's Siri, Amazon's Alexa or Google Now. It's in your satellite navigation system and may on some instant translation apps. These AI algorithms recognise your speech, provide search results and help to sort your emails and recommend some things according to your desires.

Well, maybe we aren't able to teach you how to create an AI from the ground yet, however we can show you how to create a limited voice assistants using just predefined voice commands with the help of the Artyom.js library in your ReactJS project.

Requirements

Whatever the local server that you are using to test your application, like webpack, it needs to implement a secure protocol (https), otherwise you will need to allow the usage of the microphone everytime Artyom restarts itself to run in continuous mode.

Besides, remember that Artyom.js only works in Google Chrome as this is the only browser that supports the mentioned APIs. You will need obviously a microphone too if you are willing to add voice commands (you can simulate commands if you don't have any microphone).

1. Install Artyom.js

Artyom.js is an useful wrapper of the speechSynthesis and webkitSpeechRecognition APIs. Besides, it also lets you to add voice commands to your website easily so you can build your own Google Now, Siri or Cortana with predefined voice commands.

To install this library, switch to the directory of your project with the terminal and install it via NPM executing the following command:

npm install artyom.js

The library is written in TypeScript but it's transpiled into a JavaScript build that can be used from any JS framework like ReactJS. For more information about this library, please visit the official repository at Github here or the documentation here.

2. Create a voice commands loader class

When working with Artyom.js, the voice commands are very easy to implement. A command is a literal object with at least 2 properties namely indexes and action, where indexes is a simple array with strings that will trigger the command (action) if the spoken text by the user matches any of them.

Start by creating a new file namely ArtyomCommands.js, here you will add all the commands that you want to add to Artyom. The structure that we'll use to inject the commands within an instance of Artyom will be the following:

// ArtyomCommands.js
export default class ArtyomCommandsManager {

    // The ArtyomCommandsManager class expects as argument in the constructor
    // an already declared instance of Artyom.js
    constructor (ArtyomInstance){
        this._artyom = ArtyomInstance;
    }
    
    // Execute the loadCommands method to inject the methods to the instance of Artyom
    loadCommands(){
        let Artyom = this._artyom;

        // Here you can load all the commands that you want to Artyom
        return Artyom.addCommands([
            {
                indexes: ["Hello", "Hi"],
                action: () => {
                    Artyom.say("Hello, how are you?");
                }
            },
            {
                indexes: [/How are you/, /Regular expressions supported/],
                smart: true,
                action: () => {
                    Artyom.say("I'm fine, thanks for asking !");
                }
            },
            {
                indexes: ["Generate reports of * of this year"],
                smart: true,
                action: (i, month) => {
                    let year = new Date().getFullYear();
                    
                    Artyom.say(`Generating reports of ${month} ${year} `);

                    Artyom.say("Ready ! What were you expecting? write some code you lazy bear !");
                }
            },
        ]);
    }
}

The helper class ArtyomCommandsManager expects an instance of Artyom as argument in the constructor. The execution of the loadCommands method of the class, will inject the there declared commands in the given instance of Artyom. Note that this is just a scalable approach, as you can add the commands dinamically to artyom wherever you want.

3. Creating your basic assistant

Now that we have some commands to add in Artyom, we can now proceed to initialize it. Using the advantages of the State in React, we are going to create 3 simple properties that we'll handle the status of buttons and actions in our very simple React application. 2 Boolean flags that will notice when artyom is already recognizing commands and spoking and a text variable that stores the text of a textarea where some text can be typed and spoken by Artyom.

With the context of the following class, Artyom needs to be exposed to the entire class, therefore we have declared a constant named Jarvis that stores an instance of Artyom. Our markup is very simple too, there are 2 simple buttons that starts and stop the command recognition. The actions of this buttons are binded to the startAssistant and stopAssistand methods of the class that will execute some code to start Artyom. As the methods require the this context to update the state, you need to bind it in the constructor method of your main class, where you will inject the commands too with the previous created class. Having made that, the rest of the code is pretty straight forward and easy to read as long as you know React:

import React from 'react';

// Import the Artyom library
import Artyom from 'artyom.js';

// Import the previously created class to handle the commands from another file
import ArtyomCommandsManager from './ArtyomCommands.js';

// Create a "globally" accesible instance of Artyom
const Jarvis = new Artyom();

export default class App extends React.Component {
    constructor (props, context){
        super(props, context);

        // Add `this` context to the handler functions
        this.startAssistant = this.startAssistant.bind(this);
        this.stopAssistant = this.stopAssistant.bind(this);
        this.speakText = this.speakText.bind(this);
        this.handleTextareaChange = this.handleTextareaChange.bind(this);

        // Prepare simple state
        this.state = {
            artyomActive: false,
            textareaValue: "",
            artyomIsReading: false
        };

        // Load some commands to Artyom using the commands manager
        let CommandsManager = new ArtyomCommandsManager(Jarvis);
        CommandsManager.loadCommands();
    }

    startAssistant() {
        let _this = this;

        console.log("Artyom succesfully started !");

        Jarvis.initialize({
            lang: "en-GB",
            debug: true,
            continuous: true,
            soundex: true,
            listen: true
        }).then(() => {
            // Display loaded commands in the console
            console.log(Jarvis.getAvailableCommands());

            Jarvis.say("Hello there, how are you?");

            _this.setState({
                artyomActive: true
            });
        }).catch((err) => {
            console.error("Oopsy daisy, this shouldn't happen !", err);
        });
    }

    stopAssistant() {
        let _this = this;

        Jarvis.fatality().then(() => {
            console.log("Jarvis has been succesfully stopped");

            _this.setState({
                artyomActive: false
            });
            
        }).catch((err) => {
            console.error("Oopsy daisy, this shouldn't happen neither!", err);

            _this.setState({
                artyomActive: false
            });
        });
    }

    speakText() {
        let _this = this;

        _this.setState({
            artyomIsReading: true
        });

        // Speak text with Artyom
        Jarvis.say( _this.state.textareaValue, {
            onEnd() {
                _this.setState({
                    artyomIsReading: false
                });
            }
        });
    }

    handleTextareaChange(event) {
        this.setState({
            textareaValue: event.target.value
        });
    }

    render() {
        return (
            <div>
                <h1>Welcome to Jarvis Assistant</h1>

                <p>In this very basic assistant, you can say hello and ask for some reports e.g `Generate report of April of this year`</p>
                
                {/* Voice commands action buttons */}
                <input type="button" value="Start Artyom" disabled={this.state.artyomActive} onClick={this.startAssistant}/>
                <input type="button" value="Stop Artyom" disabled={!this.state.artyomActive} onClick={this.stopAssistant}/>

                {/* Speech synthesis Area */}

                <p>I can read some text for you if you want:</p>
                
                <textarea rows="5" onChange={this.handleTextareaChange} value={this.state.textareaValue}/>
                <br/>
                {/* Read the text inside the textarea with artyom */}
                <input type="button" value="Read Text" disabled={this.state.artyomIsReading} onClick={this.speakText}/>
            </div>
        )
    }
}

Too much for "assistant" isn't? It doesn't implements a lot of commands, however that is up to you and what you want to achieve with this library. Note that of Artyom.js, we only used the Artyom.say and Artyom.initialize method and there other methods available for this library, for example to redirect the recognized text by the user, to create your own dictation application etc, so don't forget to visit the repository of the library for more tips and features.

Happy coding !