FindMyWheels

The Bot Going Around
The App's Loading Screen
The App at Work (Capturing Images)
The Backbone of Our Project (Backend)
The Frontend (Looking Good!)
The Data Stored in Firebase

Inspiration

Our inspiration came from many sources: our own struggles with cars in parking lot, our love for bots and AI, and literally every sitcom where they have trouble finding their parking spot. We decided to solve an age-old problem by letting our code remember our spot for us while not requiring any changes to the current parking structures.

What it does

There are two fundamental aspects of the solution. Analyzing the parked car images to identify the car & parking location & automating the process of taking images. For the project, we focused on the ML algorithm, front-end and back-end to analyze the images & figure out the parking location. However, we have also thought through and implemented a pilot solution to automate the ‘image-taking’. We built a mini robotic car running with Arduino using a line following algorithm to follow a line around the entire parking lot, with an Android app taking pictures of cars every time it sees one. We then use the power of the Gemini Pro Vision model (which we heavily tested on pictures of cars) which recognises and returns the car's color, license plate number, brand and model name. All this information then gets sent to Firebase Realtime Database which can be accessed by the front-end dashboard, allowing users to search up their license plate number and instantly find their parking spot.

How we built it

It was a four-part process, starting with training and testing the model on relevant data. Initially we tried training a VertexAI model on our data, but Gemini gave faster and more accurate results. We then built an API system using FastAPI in order to connect the android app and the backend effectively. Then we connected our backend to Gemini for image recognition, processing and returning data before moving on to designing the android app used to take the pictures. Finally, we placed our phone on the bot, paired it with the HC-05 Bluetooth Module to connect Arduino to Android. We incorporated infrared (IR) sensors to detect black lines on the parking surface, signalling the bot to halt its movement and triggering the app to capture images of the parked cars at designated spots.

Challenges we ran into

One of our most significant hurdles was transforming a complex concept into a fully functional prototype with limited experience in machine learning and image processing. Our initial approach involved gathering a vast amount of data manually—photographing cars in parking lots (always with permission!)—and experimenting with training a model on VertexAI. Although we achieved reasonable results, the costs were prohibitive, and the performance didn't meet our expectations.

Switching to the Gemini Pro Vision model marked a turning point, offering us not only superior accuracy but also speed, crucial for real-time applications.

However, integrating all components posed its own set of challenges. Determining an effective method for the Arduino-controlled car to know precisely where to stop and capture images required innovative thinking. We eventually implemented a line-following algorithm that was both effective and adaptable. This allowed us to direct the robotic car to specific stopping points by changing the color of the line it followed. This approach was simpler and more reliable than processing continuous video feed, which would have been significantly more complex and resource intensive.

Through these challenges, we not only developed a working prototype but also gained invaluable insights into practical AI application, hardware-software integration, and real-time data processing within a constrained budget and timeline.

Accomplishments that we're proud of

We've achieved a remarkable feat by seamlessly integrating various technologies and components, including machine learning, Android development, Arduino hardware, and real-time data processing, to bring our project to life. Our ability to adapt to challenges and pivot from initial approaches, such as transitioning to the Gemini Pro Vision model for image recognition, showcases our agility and problem-solving prowess. Developing innovative methods like the line-following algorithm for precise positioning and image capture demonstrates our skills. Through this project, we've gained invaluable experience and learning opportunities in AI application, hardware-software integration, teamwork, and problem-solving. We've created a prototype with potential real-world applications, laying the foundation for further development and potential commercialization.

What We Learned

We brought together our skills in frontend, backend, API management, Android development and Hardware with Arduino. It was amazing to see how each piece fit together to make something that works in the real world. Using the Gemini Pro Vision AI technology opened our eyes to how AI can solve everyday problems like finding your car. It’s not just about cool tech—it’s about making life easier. We learned how to come together to brainstorm various solutions for a certain problem, for example, figuring out how to guide our robotic car precisely. We ended up learning a lot by trying different solutions until we found one that worked. We learned a ton about working together. When everyone knows their part and communicates well, even complex projects come together well. In all, FindMyWheels taught us how to think and work together as innovators.

What's next for FindMyWheels

Currently, FindMyWheels exists as a prototype with a smartphone serving as the camera. However, our vision extends beyond this initial stage. We intend to elevate the system by integrating advanced components, such as Raspberry Pi and a high resolution camera, to enhance its capabilities. We will then take this prototype and present it to malls and request a trial period where they can see how it works out and how people respond to it. Additionally, we can use the same algorithm (with appropriate tweaks) in warehouse inventory management, retail-store display, and stocking and much more. We definitely plan to take this project ahead and convert it into a start-up.