com.webos.service.ai.voice

API Summary

This service is used to recognize voice input.

Overview of the API

(click to expand)

Before using the service, you need to set up Google Assistant on your device. 

Step A:  Create credentials.json file.

  1. ​Create a Google account. 
  2. Go to the Actions Console and perform the following steps:
    1. Create a new project.
    2. Register your device model and download the OAuth-2.0 credentials file. (See instructions.)
      Note: It is recommended that you select the device type as Auto to avoid errors during the setup. 
  3. After downloading the file,
    1. Move it from your local PC to /etc/googleAssistant/client_secret.json on your device using the command:​
      $ scp <downloaded json file> root@<your device ip>:/etc/googleAssistant/client_secret.json 
    2. From the device, run the following script:
      $ cd /etc/googleAssistant
      $ ./get_credentials.sh
  4. Go to the URL generated by the above script. It provides some code.
  5. Copy the code and paste it on the device terminal and run the code. It creates the 'credentials.json' file. 
  6. Enable Google Assistant API, and configure the consent screen and activity controls for your project/account. (See instructions.)

Step B: Register the device ID.

  1. Open device_id.json file. 
    $ cd /etc/googleAssistant
    $ vi ./device_id.json
  2. Modify the fields:
    • id - Input the device instance string. Typically, 'my_webos'.
    • model_id -  Input the device model ID registered in Step A.
  3. Register the device using the command:
    $ ./register_device_id.sh

Step C: Input the device model and device ID for AI service.

  1. Enter the command:
    $ vi /var/systemd/system/env/ai.env
  2. Modify the fields:
    • GOOGLEAI_DEVICE_MODEL - Input the device model ID registered in Step A.
    • GOOGLEAI_DEVICE_ID - Input the device instance string registered in Step B.

Step D: Reboot device or restart service daemon.

  • Enter the command:
    $ reboot
    OR,
    $ systemctl restart ai

Step E: Launch the service using the start() method.

 


Open All


start

Description

Starts the service based on the mode selected. It also checks if speech recognition is to be activated by a keyword or a press-to-talk button.

Note: This method is only for allowed clients.

Parameters

Name

Required

Type

Description

modeRequiredString

Select the service mode. Possible values are: 

  • continuous - The service will restart automatically when a conversation ends.
  • single - The service will be stopped when a conversation ends.
keywordDetectRequiredBoolean

Indicates if a keyword is to be used for activating speech recognition. Possible values are:

  • true: Activated by a keyword. The default keyword is "Snowboy". 
    To change the keyword model, refer snowboy.kitt.ai. Download custom the keyword model and move it to /etc/googleAssistant/ on your device. Then, modify KEYWORD_DETECT_MODEL in /var/systemd/system/env/ai.env.
  • false: Activated by a button press.

Call Returns

Name

Required

Type

Description

returnValueRequiredBoolean

Indicates the status of operation. Possible values are:

  • true - Indicates that the operation was successful.
  • false - Indicates that the operation failed. Check the "errorCode" and "errorText" fields for details.
errorCodeOptionalNumber (int8_t)

The error code for the failed operation.

errorTextOptionalString

Indicates the reason for the failure of the operation. See the "API Error Codes Reference" section for details.

Example

# luna-send -n 1 -f luna://com.webos.service.ai.voice/start '{"mode": "continuous", "keywordDetect": true}'

{

"returnValue": true

}

 

# luna-send -n 1 -f luna://com.webos.service.ai.voice/start '{"mode": "single", "keywordDetect": false}'

{

"returnValue": true

}


getState

Description

Gets the current state of the service. It can have the following states:

  • idle: Before start.
  • waitActivation: Wait for keyword activation.
  • recording: Uploading audio to the server for speech recognition.
  • thinking: Waiting for results from server.
  • answering: Answering to the request through the speaker.

 

Parameters

Name

Required

Type

Description

subscribeOptionalBoolean

Subscribe for notifications. Possible values are:

  • true - Get notifications.
  • false - Notifications are not required.

Default value: false

Call Returns

Name

Required

Type

Description

returnValueRequiredString

Indicates the status of operation. Possible values are:

  • true - Indicates that the operation was successful.
  • false - Indicates that the operation failed. Check the "errorCode" and "errorText" fields for details.
subscribedOptionalBoolean

Indicates if subscribed to get notified. Possible values are:

  • true - Get notifications
  • false - Notifications are not required
stateRequiredString

Indicates the running state of the service. Possible values are:

  • idle
  • waitActivation
  • recording
  • thinking
  • answering
errorCodeOptionalNumber (int8_t)

The error code for the failed operation.

errorTextOptionalString

Indicates the reason for the failure of the operation. See the "API Error Codes Reference" section for details.

Example

# luna-send -n 1 -f luna://com.webos.service.ai.voice/getState '{}'

{

"returnValue": true

"state": idle

}

 

# luna-send -i -f luna://com.webos.service.ai.voice/getState '{"subscribe": true}'

{

"returnValue": true

"state": "idle"

}

{

"state": "waitActivation"

}

{

"state": "recording"

}

...


getResponse

Description

Returns speech recognition results. This method is asynchronous.

Parameters

Name

Required

Type

Description

subscribeRequiredBoolean

Subscribe for notifications. Possible values are:

  • true - Get notifications.
  • false - Notifications are not required.

Default value: false

Call Returns

Name

Required

Type

Description

returnValueRequiredBoolean

Indicates the status of operation. Possible values are:

  • true - Indicates that the operation was successful.
  • false - Indicates that the operation failed. Check the "errorCode" and "errorText" fields for details.
subscribedOptionalBoolean

Indicates if subscribed to get notified. Possible values are:

  • true - Get notifications.
  • false - Notifications are not required​.
providerOptionalString

Describes where the results of speech recognition are from.

It specifies the AI service provider, for example, googleassistant

responseOptionalString

This value consist of speech recognition results.

Results formats maybe different by provider.

Example

# luna-send -i -f luna://com.webos.service.ai.voice/getResponse '{"subscribe": true}'

 

Returns:

{

"returnValue": true

}

 

Googleassistant for webOS OSE case

Example subscribe for Trigger Word Detection:

{

    "provider": "googleassistant",

    "response": {

        "keywordDetected": 1

    }

}

 

Example subscribe for Automatic Speech Recognition:

{

    "provider": "googleassistant",

    "response": {

        "partial": "what"

    }

}

{

    "provider": "googleassistant",

    "response": {

        "partial": "what's"

    }

}

{

    "provider": "googleassistant",

    "response": {

        "partial": "what's the"

    }

}

{

    "provider": "googleassistant",

    "response": {

        "partial": "what's the weather"

    }

}

{

    "provider":"googleassistant",

    "response":{

        "displayText":"It looks pleasant outside Currently in Seoul it's 79 and partly cloudy. The forecast today is 78 and mostly cloudy. ( More on weather.com )"

    }

}


stop

Description

Stops the service. This method is only for allowed clients.

Parameters

None

Call Returns

Name

Required

Type

Description

returnValueRequiredBoolean

Indicates the status of operation. Possible values are:

  • true - Indicates that the operation was successful.
  • false - Indicates that the operation failed. Check the "errorCode" and "errorText" fields for details.
errorCodeOptionalNumber (int8_t)

The error code for the failed operation.

errorTextOptionalString

Indicates the reason for the failure of the operation. See the "API Error Codes Reference" section for details.

Example

# luna-send -n 1 -f luna://com.webos.service.ai.voice/stop '{}'

{

"returnValue": true

}


ERROR REFERENCES

Error Code

Error Text

Error Description

0Not error

Not error.

-1Unknown error

Unknown error.

-2Invalid parameters

This error occurs when api parameters are incorrectly.

-3Invalid configurations

This error occurs when system missed the environment files.

-5Network error has occurred

Network error has occurred.

-6Not supported

Not supported api.

-7Service had not started

Service had not started.

-8Service had already started

Service had already started.

-9Microphone error has occurred

Microphone error has occurred.

-10Audio error has occurred

Audio error has occurred.

-11Camera error has occurred

Camera error has occurred.


Except as noted, this content is licensed under Creative Commons Attribution 4.0 and sample code is licensed under Apache License 2.0.