Gemini Cooking

Generate recipes with ingredients on hand via Vision and also pantry management.

Comment

Inspiration

I’m a dad of 2 (4 and 9) and I know basic cooking skills. I always run into the ‘What am I going to cook today’ and inspiration with what I have on hand is not always there. I know some people have restriction while other preferences so decided to create something fun and useful that would account for all that.

What it does

Identifies ingredients from picture provided: This is done via Gemini Vision and add those ingredients to the pantry.

  • Generate recipes: Generate recipes based on given ingredients accounting for dietary preferences as well as potential allergens. As a bonus also generate recipes with common ingredients in the event you may have them but have not added to the app. This is currently done using Gemini 1.0 Pro API because even when Gemini 1.5 Pro provides considerably better results, it’s not as fast nor always available.
  • Ask the chef: Let users ask questions to chef on regards to recipe and cooking in general. This adds a layer of convenience by letting users clear doubts directly in the recipe.
  • Pantry Management: Conveniently allow users to keep track of what’s in the pantry as well as shopping list.
  • Chef Personas: Add a layer of fun yet convenience when crafting recipes for users.
  • Recipes Book: Holds all generated recipes for easy future access.

How we built it

I used FlutterFlow which is a low code platform for app development using Flutter. Additionally used the Gemini APIs for Vision and Text Generation.

Challenges we ran into

Gearing Gemini to provide consistent structured data was by far the hardest thing. The next one is the current speed and availability of the API because 1.5 model perform considerably better however the speed is still slow. As models become faster and more knowledgable the app will become better at helping users.

Accomplishments that we're proud of

Being able to materialize and having systems talk to each other successfully. Learned a lot in the process and even think I can have this app deployed in the Google Playstore and Apple AppStore soon.

What we learned

Everything! As I’m not a full stack developer I learned about LLMs, GoogleAI Studio, RAGs, JSON schema, Function Calling, API structure, App State Management, Design Systems, algorithm design and even FlutterFlow as a platform.

What's next for Gemini Cooking

  • Better API error handling
  • Deployment to both the AppStore and PlayStore.
  • InApp Purchases and Subscriptions packages setup.
  • Cooperation so couples and friends can plan together.
  • Sharing on socials so users can spread the word.
  • UI Enhancements
  • Ingredient Management Logistics (Pantry, Shopping List, Active Search, Used Ingredients)
  • InstaCart Integration to let users plan ahead and have ingredients delivered.
  • MyFitnessPal Integration to let users keep track of their macros.
  • Social Features within the app to foster community.
  • Many more chefs.
  • Gamification Elements and more.

Videos

Built With

Share this project:

Updates