Alexa Smart Home Skill Tutorial
We are going to make a skill which switches a binary state on a Raspberry Pi by Alexa. In this article I focus only on the "missing links" between the found documents to get a skill up and running without any security consideration. All program code are in Node.js.
Overview: Data flow after skill activation
- Alexa Service takes care of the voice recognition and creation from text. It also maintains the Account Linking - the mapping of user credetentials from Alexa to your service. You work on this layer from your Amazon Developer Console
- Your Lambda Function on AWS gets the commands and credetentials to Your Cloud Service through a JSON API, communicates with Your Cloud Service and formulates the response for the user back to the Alexa Service. You work on this layer from your AWS (Amazon Web Services) Console
- Your Cloud Service knows the users and their devices registered to the service.
- The Code on Your Raspberry Pi switches something.
Steps to Create a Smart Home Skill
This document will be our guideline but we extend it on the following points:
- We make a dummy OAuth 2.0 implementation on Your Cloud Service.
- We make an IAM role to let change the Virtual Private Cloud setup.
- We make a Virtual Private Cloud which allows outbound internet access.
- We make a simple Lambda Function which communicates with Your Cloud Service.
- We make Your Cloud Service capable to get connections from devices behind a firewall/NAT/router making it an MQTT broker.
- We make your Raspberry Pi capable to get commands from Your Cloud Service making it an MQTT client.
Testing the Cloud Service from an Alexa Skill needs the Cloud Service to be accessible from the Internet. You can run your Cloud Service on your developer PC by making it accessible from the internet by ngrok. It creates http(s) tunnel from an unprompted domain like https://686396a3.ngrok.io to a port on your PC. Other nice features from ngrok are
- inspecting the input/output of the tunneled port on a web interface (on http://127.0.0.1:4040 per default)
- selecting a region on the Earth as the other end of the tunnel
Follow the guideline linked above but skip the code itself in section Create a Lambda function and add code and come back as you arrive to section Provide Account Linking Information!
A Smart Home Skill needs secure authorized access to the Cloud Service
A Smart Home Skill sees itself as a voice user interface (VUI) on top of an existing cloud service. That's why there is no communication between an Echo device and the smart device at home. Not even in device discovery Except some marketing exceptions like of Philips Hue. Probably Echo Plus changes this situation but I don't know it.
The secure authorized access is accomplished with the use of OAuth 2.0. It is an open standard to delegate specific authorizations. Here is a good introduction. OAuth 2.0 Authorization Code Grant Type is a requirement of Smart Home Skill as it is immediately obvious on the skill's configuration page. Here is a detailed description how to link an Alexa user with a user in your system and which concrete data exchange sequence is expected.
We build a dummy OAuth 2.0 authorization in our cloud service.
The Cloud Service
The cloud service has to handle three issues:
- Authorization delegation to let Alexa act on behalf of the user.
- Cloud Service specific API (a simple switch in our case).
- Managing connected devices.
We skip the device management now and restrict us to the first two points. These two issues are normally related but for the sake of simplicity we make only a fake authorization just to satisfy the Smart Home Skill requirements.
'use strict';
const express = require('express');
const app = express();
const querystring = require('querystring');
//Dummy authorization with the Authorization code grant flow
//https://developer.amazon.com/docs/custom-skills/link-an-alexa-user-with-a-user-in-your-system.html#h2_login
//------------------------------------------------------------------------------------------------------------
const gdata = {
skillClientId:
'YOUR_SKILL.ACCOUNT_LINKING.CLIENT_ID.ON_THE_SKILL_CONFIGURATION_PAGE',
skillClientSecret:
'YOUR_SKILL.ACCOUNT_LINKING.CLIENT_SECRET.ON_THE_SKILL_CONFIGURATION_PAGE',
//now a fix authorization code
authorizationCode: 'NOW.A.FIX.AUTHORIZATION.CODE',
//now a fix access token
accessToken: 'NOW.A.FIX.ACCESS.TOKEN',
authorizationContext: {}
};
//THIS URL MUST BE WRITTEN TO YOUR_SKILL.ACCOUNT_LINKING.AUTHORIZATIONURL
//as https:/your_host/AuthorizationURL
app.get('/AuthorizationURL', function(req, res) {
const query = req.query;
//read expected query parameters
const state = query.state;
if (!state) {
return res.status(400).send('No state query parameter!');
}
const client_id = query.client_id;
if (!client_id) {
return res.status(400).send('No client_id query parameter!');
}
const response_type = query.response_type;
if (!response_type) {
return res.status(400).send('No response_type query parameter!');
}
const redirect_uri = query.redirect_uri;
if (!redirect_uri) {
return res.status(400).send('No redirect_uri query parameter!');
}
//verify their values
if (client_id !== gdata.skillClientId) {
return res.status(400).send('client_id is not valid!');
}
if (response_type !== 'code') {
return res.status(400).send('response_type is not valid!');
}
gdata.authorizationContext = {
redirect_uri:decoded_redirect_uri,
state
};
//send a html with a login button without real authentication
//Clicking on the login button redirects to /successfulLogin
res.send(
'<h1>Login</h1><button onclick="location.href=\'successfulLogin\';">login</button>'
);
});
//In successful login we redirect to Amazon with the got authorization context
//to finish the Account Linking.
app.get('/successfulLogin', function(req, res) {
if (!gdata.authorizationContext.redirect_uri) {
return res
.status(400)
.send('authorizationContext.redirect_uri is missing!');
}
if (!gdata.authorizationContext.state) {
return res.status(400).send('authorizationContext.state is missing!');
}
const query = querystring.stringify({
code: gdata.authorizationCode,
state: gdata.authorizationContext.state
});
res.redirect(gdata.authorizationContext.redirect_uri + '?' + query);
});
//THIS URL MUST BE WRITTEN TO YOUR_SKILL.ACCOUNT_LINKING.ACCESS_TOKEN_URI
//as https:/your_host/AccessTokenURI
app.post('/AccessTokenURI', function(req, res) {
const query = req.query;
//No checks for
// * client_id
// * code (for firts token access)
// * refresh_token (for new access token)
res.set({
'Content-Type': 'application/json;charset=UTF-8',
'Cache-Control': 'no-store',
Pragma: 'no-cache'
});
res.send({
access_token: gdata.accessToken,
token_type: 'Bearer',
expires_in: 2419200, // a month in sec
"refresh_token": "a_fix_refresh_token"
});
});
//Switch API without authorization
//--------------------------------
app.post('/setLedOn', function(req, res) {
console.log('setLedOn');
res.send();
});
app.post('/setLedOff', function(req, res) {
console.log('setLedOff');
res.send();
});
//Start server
//------------
app.listen(3000, function() {
console.log('server listening on port 3000');
});
Read the code and the skill configuration at the same time. Make an ngrok tunnel and fill the missing parts in the code and on the skill configuration. Start the code.
Start ngrok tunnelling to local port 3000:
ngrok http 3000
Test your API with curl:
curl https://YOUR_ID.ngrok.io/setLedOn
Let's turn to the lambda function and its environment!
Overview about the Virtual Private Cloud
The Virtual Private Cloud (VPC) is the networking environment of an AWS service which normally consists of more microservices. The VPC allows to define the network connections between the microservices and the internet. Each AWS Lambda Function has a VPC by default which does not allow access to the internet. To change this sadly situation we must do many steps. The first one is to get right to change the VPC.
Create an IAM role which allows to change the VPC of your lambda function
Go to your AWS console, select IAM then Roles then Create role. Click Permissions and attach AWSLambdaVPCAccessExecutionRole to the newly created role.
Go to AWS Lambda on the AWS console, select your function and set this role as the Execution role. Don't forget to save.
Create a VPC with Internet access
This is a many stepping process but this video shows and explains it well.
After you have made a new VPC go back to your lambda function, select Network and set your newly created VPC as VPC and the newly created private subnets as subnets. Save.
Write the Lambda Function
Now we put a simple code in your lambda function, which uses the Smart Home Skill API but doesn't care about security.
var http = require('http');
function log(message, message1, message2) {
console.log(message + message1 + message2);
}
//on : '/setLedOn' or '/setLedOff'
function stubControlFunctionToYourCloud(path,powerResult,request,context) {
var options = {
host: 'YOUR.CLOUD.SERVICE',
port: 80,
path: path,
method: 'POST'
};
console.log('http request : ',options.toString());
http.request(options, function(res) {
console.log('STATUS: ' + res.statusCode);
console.log('HEADERS: ' + JSON.stringify(res.headers));
var contextResult = {
"properties": [{
"namespace": "Alexa.PowerController",
"name": "powerState",
"value": powerResult,
"timeOfSample": new Date(), //retrieve from result.
"uncertaintyInMilliseconds": 50
}]
};
var responseHeader = request.directive.header;
responseHeader.namespace = "Alexa";
responseHeader.name = "Response";
responseHeader.messageId = responseHeader.messageId + "-R";
var endPoint = request.directive.endpoint;
var response = {
context: contextResult,
event: {
header: responseHeader,
endpoint: {
scope:endPoint.scope,
endpointId:endPoint.endpointId
}
},
payload: {}
};
log("DEBUG", "Alexa.PowerController ", JSON.stringify(response));
context.succeed(response);
}).end();
}
exports.handler = function (request, context) {
if (request.directive.header.namespace === 'Alexa.Discovery' && request.directive.header.name === 'Discover') {
log("DEBUG:", "Discover request", JSON.stringify(request));
handleDiscovery(request, context, "");
}
else if (request.directive.header.namespace === 'Alexa.PowerController') {
if (request.directive.header.name === 'TurnOn' || request.directive.header.name === 'TurnOff') {
log("DEBUG:", "TurnOn or TurnOff Request", JSON.stringify(request));
handlePowerControl(request, context);
}
}
else {
log("DEBUG:", "Some other request", JSON.stringify(request));
}
function handleDiscovery(request, context) {
var payload = {
"endpoints":
[
{
"endpointId": "demo_id",
"manufacturerName": "Smart Device Company",
"friendlyName": "LED",
"description": "LED Switch",
"displayCategories": ["SWITCH"],
"cookie": {
"key1": "arbitrary key/value pairs for skill to reference this endpoint.",
"key2": "There can be multiple entries",
"key3": "but they should only be used for reference purposes.",
"key4": "This is not a suitable place to maintain current endpoint state."
},
"capabilities":
[
{
"type": "AlexaInterface",
"interface": "Alexa",
"version": "3"
},
{
"interface": "Alexa.PowerController",
"version": "3",
"type": "AlexaInterface",
"properties": {
"supported": [{
"name": "powerState"
}],
"retrievable": false
}
}
]
}
]
};
var header = request.directive.header;
header.name = "Discover.Response";
log("DEBUG", "Discovery Response: ", JSON.stringify({ header: header, payload: payload }));
context.succeed({ event: { header: header, payload: payload } });
}
function handlePowerControl(request, context) {
// get device ID passed in during discovery
var requestMethod = request.directive.header.name;
// get user token pass in request
//var requestToken = request.directive.payload.scope.token;
var powerResult;
if (requestMethod === "TurnOn") {
// Make the call to your device cloud for control
stubControlFunctionToYourCloud('/setLedOn',"ON",request,context);
}
else if (requestMethod === "TurnOff") {
// Make the call to your device cloud for control and check for success
stubControlFunctionToYourCloud('/setLedOff',"OFF",request,context);
}
}
};
Replace YOUR.CLOUD.SERVICE with the ngrok domain. At this point we should be able to test against the running Code Service. Let's write a test for it! Go up on the Lambda function page and select Configure test events next to the Test button. Select Create new test event, give the name turnOn and copy-paste the following:
{
"directive": {
"header": {
"namespace": "Alexa.PowerController",
"name": "TurnOn",
"payloadVersion": "3",
"messageId": "be7fd118-c5f1-4a54-b608-952c6b67279b",
"correlationToken": "AAAAAAAAAQA5bPAVAcI5S7HIit0nI7cpcAEAAAAAAAClrZCFDxI7wpil9hZBBcbEwkVoQLjwezzEi5lJ6ROs2vvF+aWuR84sGia/bLKFtgqVuK8dMbfFjgir082XNmbjxIzzHzWAoVM5Gb6gvODRC1gxwRrGKrgLfyev+kZr6MntrKpOb4AheDiYQJM5W3HvBFcLMJBm0AjDsWcebKorU8qLdfpYopCpprx0/fn47c0JJBVx0kBqJQ3uR7nQiaGe/OjH6GlSrwrVbYNUW2jQJmgOdHm0vkKv3JJpaoCtuSMTeSp9W6WcJFK1RXLm5dIO0eGNUBdIVW/iOa8eGAFCzUCikLtLZ0DnRlvXAJlFgU8gJM4PGp3+nqZ+i3LXMAhLAAIVlTdIYadQ225Yn9X7zSUlt6gBmvaTWAcdsy4SL1gNfcLSBaO6SfQ6YYZQ0TmmHg10rFMBlVIMxk/Fl//lIzYIUhEnPbg5x08bmtOy A xckbH8fTEblWJ1VOHCnLj4pEMaiqFOagJU3PY/o9WmI+Q=="
},
"endpoint": {
"scope": {
"type": "BearerToken",
"token": "access.token.4jpxOvVQlHwDVcUv0GcW44GQqRmxMflxxmgjp"
},
"endpointId": "demo_id",
"cookie": {
"key1": "arbitrary key/value pairs for skill to reference this endpoint.",
"key2": "There can be multiple entries",
"key3": "but they should only be used for reference purposes.",
"key4": "This is not a suitable place to maintain current endpoint state."
}
},
"payload": {}
}
}
Click Create and try it!
You can see the log of your skill any time by clicking the Monitoring tab then the View logs in CloudWatch link.
Activate the skill at your Alexa App
Start the Alexa App on your smartphone. Go to Startpage > Skills > Your Skills and activate your newly created skill! It should be displayed with a little green dev label. In activation Alexa tries to make the Account Linking through our OAuth 2.0 interface on the Cloud Service. After it you should be able to switch the state on your Cloud Service with a spoken command like "Alexa, turn on LED!". LED is the friendlyName defined in lambda function. I hope it works!
Setup the Raspberry Pi for Development
I assume you can log into your Raspberry Pi and have the newest Node.js installed. And now some usefull tipps to make a pleasant development environment on your Pi.
Debug remotely from Chrome
Start the node program on the Raspberry Pi:
node --inspect-brk some.js
Start ssh tunneling on your developer PC:
ssh -N -L 9229:127.0.0.1:9222 pi@ip_of_pi
This makes a bidirectional ssh tunnel saying that the host:port of 127.0.0.1:9222 on the Raspberry Pi will be mapped to 0.0.0.0:9229 on your developer PC. 127.0.0.1:port is the loopback address seen only from within the given machine. 0.0.0.0:port is visible from outside.
Go to chrome://inspect in Chrome on your developer PC and you should be able to debug your program running on your Raspberry Pi.
Mount your home directory on the Raspberry Pi to your developer PC
On your developer PC install sshfs and make a mount directory
sshfs pi@ip_of_pi: YOUR/MOUNT/DIR
You can unmount it with
sudo umount YOUR/MOUNT/DIR
Attaching the Raspberry Pi to the Cloud Service
We will use the MQTT machine-to-machine connectivity protocol with the Mosca broker and the async-mqtt client implementations. The Cloud Server will be an MQTT broker, the Raspberry Pi an MQTT client.
MQTT allows us to attach the Raspberry Pi from behind a firewall/NAT/router arbitrary long to the Cloud Service on the Internet and get commands from it.
Mosca needs a database to store the connection data. The following code snippets expects MongoDB. This Cloud Service code is not combined with the OAuth implementation seen above. It shows only a minimal MQTT pub/sub infrastructure. Bringing the two features together stays a homework for the reader. The implementation does not even distinguish between the different clients, it sends the setLedOn/Off commands to all.
'use strict';
const express = require('express');
const app = express();
const mosca = require('mosca');
if (!process.env.MONGO_URL_WITHOUT_DB) {
console.log('MONGO_URL_WITHOUT_DB environment variable must be set!');
process.exit(1);
}
if (!process.env.MOSCA_PORT) {
console.log('MOSCA_PORT environment variable must be set!');
process.exit(1);
}
//MQTT broker
//-----------
var server;
function initMqttBroker(mongoUrlWithoutDb, moscaPort) {
var ascoltatore = {
//using ascoltatore
type: 'mongo',
url: mongoUrlWithoutDb + '/mqtt',
pubsubCollection: 'ascoltatori',
mongo: {}
};
var settings = {
port: moscaPort,
backend: ascoltatore
};
server = new mosca.Server(settings);
server.on('clientConnected', function(client) {
console.log('client connected', client.id);
});
// fired when a message is received
server.on('published', function(packet, client) {
console.log('Published', packet.payload);
});
server.on('ready', function setup() {
console.log('Mosca server is up and running');
});
}
function publish(topic, payload) {
var message = {
topic,
payload,
qos: 1, // 0, 1, or 2
retain: false // or true
};
server.publish(message, function() {
console.log('publish "', message.topic, message.payload, '" done!');
});
}
function publishSetLedOn() {
publish('setLedOn', 'true');
}
function publishSetLedOff() {
publish('setLedOn', 'false');
}
//Switch API without mapping user to device
//-----------------------------------------
app.post('/setLedOn', function(req, res) {
publishSetLedOn();
res.send();
});
app.post('/setLedOff', function(req, res) {
publishSetLedOff();
res.send();
});
//Start server
//------------
initMqttBroker(
process.env.MONGO_URL_WITHOUT_DB,
parseInt(process.env.MOSCA_PORT)
);
app.listen(3000, function() {
console.log('server listening on port 3000');
});
After you combined the authorization delegation with the MQTT broker you should start a MongoDB on its default port (27017) and you can start the Cloud Service:
MONGO_URL_WITHOUT_DB=mongodb://localhost:27017 MOSCA_PORT=1883 node index.js
Make an MQTT client from the Raspberry Pi
'use strict';
const express = require('express');
const app = express();
const mqtt = require('async-mqtt');
if (!process.env.MQTT_URL) {
console.log('MQTT_URL environment variable must be set!');
process.exit(1);
}
//MQTT client
//-----------
function logError(e, title) {
console.log('error in', title);
console.log(' message', e.message);
console.log(' stack:', e.stack);
}
function initMqttClient(mqttUrl) {
var client = mqtt.connect(mqttUrl);
async function onConnect() {
try {
console.log('connected to mqtt broker');
await client.subscribe('setLedOn');
} catch (e) {
logError(e, 'onConnect:');
}
}
async function onMessage(topic, payload) {
try {
console.log('topic:', topic);
// payload can be a Buffer
console.log('payload:', payload.toString());
if (topic === 'setLedOn') {
await onSetLedOn(payload.toString());
}
} catch (e) {
logError(e, 'onMessage');
}
}
async function onSetLedOn(payload) {
try {
switch (payload) {
case 'true':
console.log('setLedOn');
break;
case 'false':
console.log('setLedOff');
break;
default:
throw 'onSetLedOn: Unknown command ' + payload;
}
} catch (e) {
logError(e, 'onSetLedOn');
}
}
client.on('connect', onConnect);
client.on('message', onMessage);
};
//Start server
//------------
initMqttClient(process.env.MQTT_URL);
app.listen(3000, function() {
console.log('Listening on port 3000');
});
You can start the MQTT client on the Raspberry Pi. We assume 1883 as the MOSCA_PORT of the Cloud Server.
MQTT_URL=mqtt://YOUR_ID.ngrok.io:1883 node index.js
After this you should be able to log "setLedOn" on your Raspberry Pi saying "Alexa, turn on LED!".