Inspiration
One of our team members regularly purchases canned soup, and she ran out without realizing it. She wondered if she could create something to automatically order more food when she ran out.
What it does
Diffy uses computer vision and OpenCV and RoboRealm to identify visual changes, to alert users when something is no longer there. Our demo involves identifying when things are missing from a pantry, so that new food can be automatically ordered.
How we built it
We first constructed a demo shelf, stocked with standard food items, and set up a live webcam feed. Then, we used OpenCV and RoboRealm to separate the output of the webcam into distinct frames and analyze sequential frames for visual changes, correcting for minute changes that we don't care about as well as changes in lighting. We then used Microsoft's Cognitive Services API for image recognition, the result of which is sent to a text-to-speech package to be read aloud. Finally, we built an Arduino with an ultrasonic distance sensor, so that the camera's output is only used and analyzed when motion is detected (the Arduino also helps the system avoid detecting a change when a person is between the camera and the shelf.
Challenges we ran into
Separating the live webcam video feed into discrete images; Putting the various independent components together; Setting up the picture such that the Microsoft Cognitive Services API can recognize it
Accomplishments that we're proud of
Getting the project to run successfully within 24 hours
What we learned
How to think in terms of processing an image in CV
What's next for Diffy
Porting Diffy from a computer into an online server, such that a low-power WiFi-connected camera can be set up anywhere
Applications
Logistics and item tracking; Detecting theft; Medical cabinets; Restocking supplies; Food notification at events (like HackHarvard!)
Log in or sign up for Devpost to join the conversation.