Interactive Christmas Tree

Firstly, Why!?

I make projects for YouTube that anyone can interact with, e.g. internet controllable remote control cars, internet playable real life rocket league. This was my Christmas special!

My channel can be found here

Excuse the clickbait title (but it is YouTube afterall)

So the goals for this project?

• Make a christmas tree that anyone can control over the internet
  
• Display a live stream of the tree
• Ensure the project can scale if there was to be an influx of users
• The status of the tree should be tweeted every hour (@InteractiveXmas)

Now this project is pretty large, but it was great fun. It is mainly software based which is typically pretty boring, but I will do my best to explain it all as brief and interesting as possible!

The Web UI

The website is built on-top of React, there isnt too much to say about it, it allows users to scroll through the pixel art and change pixels, and they can see a live video stream of the tree. The site is still up but is now read-only (and now uses a still image of the christmas tree at the end of the project) link: Interactive Christmas Tree

Initially I was loading the entire database for each ornament everytime a user loaded the page, and this was extremely expensive, in both monetary cost and resources. This article by reddit was extremely helpful in terms of looking at how they kept r/place's costs down. 

The new approach for displaying the ornaments was to save each ornament to a google storage bucket every 10 minutes, then when a user loads a page, download the BMP saved in the storage bucket, and then replay all the action (stored in firestore) that had occured to this ornament since the BMP was saved. This drastically reduced the impact on realtime database, and reduced the amount of data downloaded from realtime database (which can be pretty expensive)

The code for the UI can be found here

How does the system work?

So this project makes a lot of use of Firebase. These are the services that I have used, and what I use them for:

• Authentication: Used for authenticating users (I only allowed Google OAuth signin as didn't want to handle users credentials at all)
• Realtime Database: Stores the status of each ornament and the lights, clients can create listeners to certain areas of the database and recieve a ping when something changes, therefore updating users pages in realtime
• Functions: Runs snippets of code in the cloud (you are only charged per function call instead of all the time like a traditional server), this was used for a lot of things, e.g. Creating an array in firestore when a user creates an account to store all of their modifcation (enforce cooldowns), get entire ornament as a BMP, modify a light, modify an ornament pixel, and I think maybe a few more!
• Firestore: Stores a document for each modification to an ornament or light, this is used to track all of the changes to the system, each record had the following format:
• Where squareId is the id of the pixel on an ornament or the light on the christmas lights, and UID is the user ID.

If we look at the flow of data for changing a light on the tree:

All other clients on the page will then recieve an update from realtime database to update their client side of the UI.

The code for the infrastructure (including all of the Cloud Functions) can be found here

The Ornaments

The ornaments are very simple, they are just an ESP32 and a TFT eSPI LCD connected via a perf board (excuse the messy soldering)

I used Platform.io and the arduino framework to write the code for the ornaments, the code for that is here

The logic for the ornaments were as follows:

1. Boot up
2. Grab the full BMP image of the ornament it is configured for and store in memory
3. Display on screen
4. Listen for HTTP requests with updates to any pixels
5. Every 10 minutes repeat 2 incase a request was missed or never came through

The Lights

The lights follow the same logic flow as the ornaments, the code is equally simple: code

The Video Stream

This was an absolute nightmare, and something I really want to dig deeper into and get a better understanding of, realtime video streaming drives me insane (so if you know a lot, or anything in-fact, please reach out to me, I would love to have a conversation with you!!!)

The first thing to decide was the codec and protocol I wanted to use, I ended up going with LL-HLS, which had a latency of around 2-5 seconds, but had very low bandwidth consumption and could be displayed in browsers natively in Safari and in other browsers with use of the Media API. I am still really unhappy with a 2-5 second latency,this does not feel realtime, the other alternative was WebRTC, which on paper is a really simple concept, but for whatever reason I seem to massively overcomplicate it when trying to implement video streaming. But thats a topic for another day. rtsp-simple-server was an absolute life-saver, I installed it on an old laptop running ubuntu, and an RTSP stream was ready to go... but, my home network only has around 10mb/s upload speed, and remember the goal of must be scalable. If I had maybe 10-20 connections, my home network would be too congested and things would start going bad. This was not going to be acceptable. And also... RTSP streams cant be displayed on browsers...

Thankfully rtsp-simple-server has a proxy mode, this would listen to an RTSP stream and proxy the stream, but it would also offer the stream in LL-HLS format! This was perfect... I could have proxies in the cloud that would maintain one connection to my home network and proxy the stream. I put this behind a load balancer so if there was a spike in requests/CPU utilisation, another box would start and share the load between users! 

There was one slight bug, every hour the proxies would stop offering the LL-HLS stream, I am still unsure why, but I have opened a issue on the rtsp-simple-server github repo, and fingers crossed it can get investigated. I'd love to be able to help them but my knowledge of video streaming is still pretty bad!

Conclusion

I think thats everything... I am more than happy to answer questions about the project! Fingers crossed Ill do it again at the end of this year on a bigger scale... who knows!

Some nice analytics for the entire project (1st Dec - 25th Dec):

1602 total accounts (my target was 500!)

29,628 changes of the Christmas lights!

400,874 pixel art updates!

There are also timelapses for each ornamnet that can be viewed on the twitter (@InteractiveXmas) - be warned these are NSFW (The internet is a interesting place) Moderating this project is a whole other kettle of fish lol.

Overall the project cost me around £150, which I think is pretty fair, £100 was just the video stream as they were running on traditional VM's.