Fridge2Fit

Inspiration

Driven by my mother's role as the esteemed director of Obesity Medicine at the University of Pennsylvania, I have been deeply moved by the stories and challenges faced by her patients. These experiences have fueled my passion to delve into the field and equip everyday individuals with the knowledge and tools to proactively minimize their risk of obesity.

What it does

Upon launching our innovative web platform on their mobile device, users simply capture a photo of their refrigerator contents. In a matter of seconds, our recognition technology meticulously identifies and catalogues the ingredients. The platform then organizes these items systematically, providing users with tailored suggestions to complement their existing stock. Leveraging this inventory, our system generates an array of nutritious recipes, all carefully selected to prioritize the user's well-being and support a healthy lifestyle.

How we built it

We utilized express, a webserver designed in node.js. We utilized the AICook dataset using a model trained on RoboFlow and used groq for API access to the llama-8b-8192 model.

Challenges we ran into

Embarking on my initial Express project, we faced a series of challenges. To avoid Cross-Origin Policy Referrer errors, we had to implement server-side request handling. Moreover, while generative AI API access often comes with a price tag, we managed to find an affordable alternative in groq. However, a significant hardware challenge arose with the Raspberry Pi 4. The main issue wasn't the lack of a built-in LED display but rather the proximity of the device to the items in the fridge, which made it difficult to capture clear images. This limitation necessitated the use of additional cameras to ensure proper visibility and accurate ingredient recognition, complicating the process of turning the Pi into a functional smart fridge.

Accomplishments that we're proud of

Among our proudest achievements, we successfully completed our first project as a team at a hackathon, marking a significant milestone as it's our third hackathon together and the first time we've managed to see a project through to the end. Leveraging advanced AI models, we developed a prototype with the potential to make a real impact in the field of obesity medicine. This accomplishment not only demonstrates our technical prowess but also our commitment to addressing critical health issues.

What we learned

Throughout this project, we gained invaluable experience in developing and deploying a Node.js application, a skill set that is highly practical in the professional workspace, enabling us to adapt swiftly to evolving requirements. Initially, our Raspberry Pi concept proved infeasible, but this pivot taught us the importance of flexibility and resourcefulness. Additionally, we honed our skills in working with APIs and leveraging various AI tools, including GitHub Copilot, which played a substantial role in enhancing our work and streamlining our development process.

What's next for Fridge2Fit

What's next for Fridge2Fit is an ambitious expansion of our dataset to enhance the accuracy of our predictions. Currently, we have a collection of 5,000 images, but we aim to significantly increase this number. Additionally, we're focused on refining the user interface with proper styling and improving cloud support to ensure seamless accessibility and the ability to retrieve past recipes. As we scale and our team grows, we anticipate migrating from Node.js to a more robust web server, such as Rocket in Rust, to meet the demands of our expanding user base and developer workforce.

Built With

• aicook
• axios
• express.js
• groq
• llama
• node.js
• roboflow