Ted The Talking Toaster

About the project

Conversations with a toaster have always been very one sided. Until now, meet Ted the talking Toaster!

Items used in this project

Hardware components

Toaster Toaster x 1
Raspberry Pi Camera Module Raspberry Pi Camera Module x 1
Google AIY Voice Kit for Raspberry Pi Google AIY Voice Kit for Raspberry Pi x 1
Raspberry Pi 3 Model B Raspberry Pi 3 Model B x 1
SG90 Micro-servo motor SG90 Micro-servo motor x 2
Pipe Cleaner Pipe Cleaner x 1

Software apps and online services

Multitool, Screwdriver Multitool, Screwdriver
Drill / Driver, Cordless Drill / Driver, Cordless
Google Cloud Platform Google Cloud Platform
Google Dialogflow Google Dialogflow
Raspberry Pi Raspbian Raspberry Pi Raspbian

Hand tools and fabrication machines

Hot glue gun (generic) Hot glue gun (generic) x 1
Soldering iron (generic) Soldering iron (generic) x 1

Story

Overview

Below an overview of the content:

  • Project video
  • Toaster
  • Electronics
  • Chat/Voice bot
  • Remo.tv
  • World domination

Project Video

Toaster

To make a talking toaster the first thing we need is, a toaster. I got this one from a friend, but thrift stores are a great place to find yourself a broken toaster with good looks.

The original toaster

The first thing we're going to do is remove as much of the internals as possible, while keeping the outside looking nice.Removing the internals

Some fiddling and the right amount of violence is needed to make this happen, but in the end you'll have plenty of space.

The cleared toaster

Electronics

With all that space, we can start filling it up. Our aim is to make a talking toaster, so that's what we wil start with.

We're going to use a Raspberry Pi 3 b+ with a Google AIY kit. This kit consists of a hat, microphone and speaker. They provide a great tutorial on how to set it all up.

For this project we've used an older version of the kit, but the newer ones will work as well.

Google AIY Kit

This covers the talking hardware for our little bread burner, but we also want him to express himself. To make this happen we're going to drill two holes in one of the front panels.

Drilled holes

Through these holes we put two little servo's, they're the start of his eyebrows.

Servo glued to the inside of the toaster

Servo glued to the inside of the toaster

Now we can wire the servo to the AIY hat, first we need to solder some pins and then we can easily attach them. There's some more info on the possible extensions below.

Hat schematics

The last offline step is to glue on some fuzzy pipe cleaners as eyebrows and googly eyes as, you guessed it, his eyes.

Looking good

And with that, we've got a very good looking little heating element. He looks the part, so he deserves a name, let's call him Ted.

Chat/Voice bot

At this point you can keep following the AIY tutorial to make the toaster your personal google assistant, or write your own custom code. In this case we opt for creating a custom chatbot, using Dialogflow. They also provide a good tutorial on how to setup a chatbot.

After some creative writing our chatbot is done, but at this point Ted is just a toaster and the chatbot doesn't do much. To change this we open up the Dialogflow chatbot API. This means we can access the chatbot from our Raspberry Pi. And now the fun begins The flow of data is as followed:

Data flow just Ted

Data flow just Ted

1The microphone picks up someone speaking and saves the audio

2-3Using some Google magic (Speech-To-Text) we extract the text from the audio

4-5This text is sent to our chatbot (Dialogflow) and is matched with an intent.After which one of the possible answers is send back to the Raspberry Pi (Ted)

6-7-8Using Text-To-Speech, the text is converted to audio and played via the speaker

You can find the full code for this attached to the article.

Ted can now listen and talk, but how is he going to show the right emotion for each response?

When fetching the response from the Dialogflow API, it also sends back the intent name. Ending every intent with "_emotion" makes sure we can extract the emotion linked with each response and move the servos accordingly.

Teds emotions

Example, our intent is called "climateChange_angry" he will extract everything after the "_".This means "angry", and then moving the servo to look angry.

Now Ted can listen, talk and look frisky, that's one small step for Ted, one giant leap for toasterkind.

Remo.tv

Ted might be a tad evil, who would think a toaster could be out for world domination? As his creators we're responsible for his well being, and since he demanded to be connected to the internet to spread his message, let's do exactly that.

Remo.tv

It turns out we're not the only ones who want to connect robots to the internet, Remo.tv is way ahead of us. This means that all we have to do is attach a Pi Camera and follow their instructions.

Ted on Remo

Now people can see Ted, but talking to him is tricky. However, Remo.tv allows you to implement your own chat handeling. So removing some parts of our previous code and adding the custom chat handler does the trick. Now the data flow is:

Remo Data Flow

1Chat message is received from Remo.tv

2-3This message is sent to Dialogflow and the response with emotion is returned.

6-7The response text is converted to audio via the Text-To-Speech.

8The audio is played via the speaker and immediately picked up by the microphone.

9This audio is sent to Remo to be broadcast to everyone who's tuned in.

World domination

Toast'en

All set, Ted can start convincing the world of his vision, and we now feel like some toast.Would you like to be converted, just follow this link.

With that, all that's left to say is: #Toaster2020!

Schematics, diagrams and documents

Remo Data Flow

Ted Data flow

Code

Remo Code

The custom chat handler for Remo.tv

Ted Code

The code for just Ted as an chatbot

Credits

Photo of 8BitsAndAByte

8BitsAndAByte

We’re Dane & Nicole, two makers that create tremendously terrible tech, which we happily share with you on our channel! https://www.youtube.com/c/8bitsandabyte

   

Leave your feedback...