High-stakes Computer Vision Simon Says Game
About the project
Step in front of the Raspberry Pi, follow its instructions. If you mess up, get slapped into oblivion by its electro-magnet powered slapping arm
Project info
Difficulty: Moderate
Platforms: Raspberry Pi, OpenCV, PlatformIO
Estimated time: 3 weeks
License: GNU General Public License, version 3 or later (GPL3+)
Items used in this project
Hardware components
Software apps and online services
Hand tools and fabrication machines
View all
Story
Somehow, YouTube events have recently seen a resurgence of the Simon Says game. So much so that I started dreaming about the game, and every time I lost, I would get slapped across the room by this weird monster.
And I've always been told that once you confront your dreams, they tend to disappear.
Let me introduce "Simon Says, Simon Slaps".
A projec that includes a Pi5 brain running a mediapipe instance for pose detection with a video feed coming from a PiCamera module. If the Pi5 detects a mismatch between the pose requested and the pose done by the player, a PVC pipe dressed up as a monster releases its accumulated rotational energy to slap the user powerfully, raising the stakes.
With a plan in mind, I wrote a script referencing the angles between my limbs.

Each of these poses were recorded 15 times, and an average of all the angles and distances were saved in JSON format with hand-tweaked tolerances to allow people of different constitutions to have their poses recognized too.

Those poses were split between 3 groups: torso, arms and legs. As such, a user could be asked to raise his/her left leg while the right arm was already up making the game more complex and fun overall. (This required impossible states to be set such as left leg up followed by right leg up)
These poses were then tested on a validating script to make sure that they were detected properly. When the user would appear on camera, each pose would return a certainty index based on how close the user was to it, then, the pose with the highest score below the threshold would be considered as detected.
With that out of the way, I still needed to program the physical aspect of the build. Because code was not going to cut it. The simples idea that came to mind was to make a cross out of PVC pipes and a solid base. The rotation of the trunk would move a "hand" that would be placed at the end of an "arm" and that would be it.

It turns out, a lot more physics are involved in these builds than expected. As soon as the arm was added, the rotation was completely out of balance and MUCH slower. So I added a large bearing around the "core" PVC pipe and a 3D printed holder for it. Largely helped the out of balance issue, but that mechanism was going to have a LOT of energy stored in it otherwise, the slap would not work properly.
So I resorted to using elastic bands, but if you've ever worked with elastic bands, you know that they're far from the best when it comes to rotational motion but that's a problem for later.
For the release of energy, I opted for an electromagnet, even though my first thought was to go for a latch mechanism of some sort, I didn't have the means to realize it. So good old electronics came to the rescue. Load the arm to its initial position, turn on the electromagnet and turn off the power when the user fails. Easy right? That's the job for a relay. When the pi 5 detects mismatch, it sends an HTTP request to another microcontroller which is running a webserver on my local network. On reception of that HTTP request, it changes the state of the GPIO pin where a relay is connected. The relay then disconnects the electromagnet from its 12V power supply.
That approach was pretty cool since I could load the arm with an iPhone shortcut triggered by voice!
The build ended up being a bit weak in its movement but a lot of fun to play with!
Leave your feedback...