Sad? Sing Along And Feel Better Instantly

About the project

When detecting someone being sad, they are encouraged to sing a karaoke song to feel better. The AIY Vision Kit is being enhanced by using an ESP8266 as small display and MQTT to show the texts.

Project info

Items used in this project

Hardware components

Google Aiy Vision Full Kit - Includes Pi Zero Wh - V1.1 Google Aiy Vision Full Kit - Includes Pi Zero Wh - V1.1 x 1
ESP8266 incl. Display ESP8266 incl. Display x 1

Software apps and online services

Arduino IDE Arduino IDE


Why is this project interesting?

It is proven by science that singing makes you happy - it releases endorphins and reduces stress.

I have always been fascinated by how we can use technology to make people smile.

So if someone shows a sad face, they should be cheered up by singing a song.I was thinking that the Vision kit could get an easy upgrade by adding a small display to it.

The ESP8266 which includes a handy OLED display and Wifi is a practical device to use for this.

MQTT is used to send messages from the Raspberry to the micro controller and the micro controller shows the song texts.

How to set it up

On the Raspberry Pi:

Install the MQTT broker and client on the Raspberry Pi (Instructions:

Go to the file "" (I attached the file as I am using it, but you will find it on the Raspberry Pi in this folder: /home/pi/AIY-projects-python/src/examples/vision/joy

Make sure to make a backup and keep the file in the same folder, so that it can be found by the respective service at startup and is called automatically.

There are two lines that need to be added to the file where the mood is detected:

os.system("mosquitto_pub -h localhost -t "aiy/message" -m "happy"")


os.system("mosquitto_pub -h localhost -t "aiy/message" -m "sad"")

You can test this by typing mosquitto_sub -t "aiy/message" on the console of the Raspberry Pi. You should see something like this, when you smile and frown and the camera:

On the ESP8266:

Open the attached "esp8266_mqtt_sing_a_song.ino" file in the Arduino IDE.

Install these libraries:

-- Adafruit SSD1306 (to use the display)

- - PubSubClient (for MQTT)

Add your wifi credentials and the IP address of the Raspberry Pi (you can use ifconfig on the console to find it out) at the top of the file.

Compile and upload it to the ESP8266.

To check, if everything is working, you can take a look at the Serial Monitor within the IDE. It should look something like this: 

The End Result:

Here is a video of what happens, when a sad faces is detected (no sound):

Where can you go from here?

There are lots of interesting enhancements that can be done. Here are some ideas for you to try:

- add sound to play either from the Raspberry Pi or from the ESP8266

- play a different song every time someone is sad

- play a song when you detect happiness!

- find other ways to motivate the person with words

- invent a game moving an icon left or right depending on if the person is smiling or frowning

- invent a face gym!

- measure the time between smiling and frowning and show high scores




Leave your feedback...