This is a project I’ve been working on (and unfortunately, mostly off) for the past couple of months. For those not familiar with the format of my posts, I talk about a lot of unrelated things in the post, but link the GitHub for those who want to clone and build the project for themselves.
If this project were complete, I would’ve pointed y’all to the GitHub, but it isn’t, so I guess we’re all stuck reading just the blog š
This project is envisioned to be a Polaroid that clicks an image (it looks like a normal Polaroid, normal click), and prints out that image, but with a roast. So it could be something like a picture of someone holding a coffee cup, captioned: “Your entire personality is just caffeine and the illusion of productivity.”
ANYWAYS.
I was gifted a Polaroid back in 2020, when I quickly discovered that Polaroid film, in fact, costs a lot. And since then, I’ve had this idea on the back burner: could you gut a (broken) Polaroid and replace the printing mechanism with something more accessible? This summer gave me the opportunity to finally get to this.
Working on this project has led to an absurd amount of good fortune being bestowed upon me.
I went thrifting with some friends at a Goodwill in SF, and I was talking to one of my friends about how badly I wanted to find an old Polaroid. And I turned left, and there it was, in all its glory, an old Polaroid. Five USD. The timing of it was genuinely insane.
Now, the compute. I was under the impression that I had two Raspberry Pi Zeros, and I spent two weeks looking for a Zero 2 W. They have been incredibly hard to find in the US. I really didn’t want to deal with a regular Zero, since the 2 W is so much more powerful, for the same form factor. Somehow, as I was cleaning out my box of electronics, I opened up the other one and realized it was a 2 W. I hadn’t touched it in two years, always dealing with the other one.
Recently, I came across this thermal printer kit from SparkFun. It exposes a TTL Serial interface, allowing you to print directly. I haven’t seen them be in stock ever since I’ve started looking, and I’m not super fond of the price point. I also think the UART makes it a bit too easy, so I decided to buy a generic cat printer (which are everywhere on AliExpress, Amazon, etc.). ChatGPT was very against it, but I found at least five repositories online where people had reverse-engineered the Bluetooth protocol for these printers. I thought it was worth the risk to buy a generic one and hope one of the repositories worked.
I used the Opuu cat printer repository. It did take me some time to get it working, but I can now succesfully print text and pictures using the printer! Here is my hello world:
And the printer generates enough thrust to be able to push the paper out through the rollers of the polaroid casing!
The picture is obtained through a RaspPi camera connected to the Pi via a flex extension cable. The question then was, how do you get a roast?
An initial idea was to connect the Pi via the internet to an LLM, but the idea of a camera having to connect to the internet to work made me really depressed. The second idea was to have a local LLM, but after running some numbers, I came to the conclusion that even the most lightweight model I could find wouldn’t really generate words quickly enough for my purpose.
And that is how I arrived at my final design.
The initial idea was fairly simple: detect objects in an image, and use those detections to select a roast from a library of pre-written responses. But I quickly realized that detecting a “person” or a “chair” isn’t nearly enough information to make a roast interesting. I wanted the camera to pick up on things like what someone was wearing, or what objects they were surrounded by.
So I ended up with a cascade of object recognition models. I used a quantized SSD MobileNetV2 model for general object detection, and if it detects a person, that triggers more specialized models: BlazeFace for face detection, MobileNetV3Small for facial attributes, and PP-LCNet for clothing attributes. This way, I don’t have to run every model on every image, which is pretty important when you’re working with a Raspberry Pi Zero 2 W.
I (well, Claude, which is also probably why the roasts are so lame) then created a very dense JSON library of pre-written roasts for ALL OBJECT CLASSES. The selection logic uses the detected objects, selects items of interest stochastically, and looks up a roast.
I think there’s a smarter, perhaps tree-based, or just cooler roast generation algorithm here, but I haven’t really experimented with that all that much. I’m taking a cognitive science class right now, and I’ve been really inspired by pre-LLM approaches to sentence generation and computational modeling that folks developed. I think there’s definitely a lot of potential there, but I haven’t really looked into it just yet.
Here’s an example of what the model can do right now. When I inputted one of my friends’ linkedin picture into it, it responded with:
“A tie is just a leash you willingly put around your own neck to convince people you have your life together.”
As it stands, my components all work individually. The camera clicks pictures, the Pi is able to generate roasts, and the thermal printer is able to print pictures. I need to put it all together.
Here’s me having gutted the Polaroid:
This took a long time since I wanted to make sure my casing wasn’t damaged, and that I wouldn’t accidentally hurt myself. This model has a flash capacitor rated at ~300V that needs to be dealt with properly. Fortunately, polaroid repair videos online helped immensely.
I intend to fit my camera, Raspberry Pi, a battery, and the internals of the thermal printer into the casing. There is plenty of space; I just need to be smart about it.
I’ll follow up soon with more updates as they come!
Leave a Reply