JourneyAI

Learn More On Specific Photo
Journey
landing page
upload photos for the journey
real time journey upload

Inspiration

Have you ever returned home from a trip and realized you've forgotten some of the most memorable parts? What if there was a way to easily relive those memories? That's why we embarked on creating JourneyAI—a solution designed to provide travelers with enriched insights into their destinations while offering a means to reminisce about their journeys.

What about the opposite? You're currently on the trip and you are curious about the landmarks and building around you. You can take a picture and start building your journey while also having a personalized journal with infinite knowledge to ride along with you in your adventure!

What it does

JourneyAI offers a unique and interactive platform for users to relive or create new JOURNEYS! As a user uploads images captured during their travels or snap new ones, their image will eventually be processed and displayed on a map where they can easily learn and explore more about the captivating destinations they've been to with ease. RELIVING THEIR JOURNEY OR MAKING NEW ONES RIGHT IN THE MOMENT.

Here's how it works.

Users will have the option to either view previous albums or create a new album.
When creating a new album, users are able to upload multiple photos or take photos real time.
Uploaded photos appear on an interactive map. Gemini generated text describes each photo in a users' journey.
Users can interact with the text by highlighting, asking questions, or simply pressing 'learn more' to learn more and ask questions about the places that they have visited.

How we built it

We used Next.js, Google's Gemini, Fetch.AI, and Firebase to accomplish all of our goals. This proved to be both difficult and convenient, as most of us have never used these technologies before! Thanks to Next.js and Fetch.ai, we built our entire frontend in React and entire backend with an agent pipeline. In terms of Fetch.AI, we were able to piece together multiple agents using a combination of their tooling like mailbox and the agentverse. That, combined with Gemini's API and Firebase APIs allowed us to make our vision come true. Our first agent is hosted on the agentverse and is in charge of making queries to start our pipelin. This agent sends a message to our preprocessing agent which reorders the photos stored in our firebase storage, and sends them one by one to our next agent. This third agent is the gemini agent, and is in charge of calling the Gemini API! We did our best to utilize the capabilities of Gemini by using the quick and detailed image processing and their version 1.5 pro. As this agent gets the initial generated text processed from the image, it sends the data to our last agent. Our last agent finally stores everything in our Firebase firestore. By effectively using Fetch.ai, we were able to create a cohesive pipeline of multiple photos to generated text and storing them in the database. On our Next.js app, we further maximized Gemini by having the ability to add on to the original text generated from the gemini agent explained earlier. This allows users to infinitely dive deeper into the places they've been and explored.

Challenges we ran into

Our biggest challenge was utilizing technologies that we were unfamiliar with. Gemini and Fetch.AI were the two major components of our project, and there was a big learning curve. One of the issues we ran into was integrating the frontend with the backend and synchronizing the frontend to receive the text results from the images after being processed by the Fetch.AI pipeline. Simpler issues like displaying the photos on an interactive map, and our time frame for design was expanded due to underestimation of what we thought wouldn't be issues.

What we learned

We decided to utilize Next.js to build out our web app that incorporated Fetch.ai, Firebase APIs, and Gemini APIs. For Fetch.AI, we learned how to utilize and connect several agents by fully understanding the Fetch.AI documentation and reaching out to the Fetch.AI team for assistance. Additionally, we learned to link the various pages together and to incorporate components into our pages in order to fully immerse the user(s) and to provide a platform for users to relive their travel experiences.

What's next for JourneyAI

With only 36 hours, our team only had enough time to create an MVP, so there were plenty of other features that we wish to add in the future. Primarily, we'd love to add the ability to see current events happening in the area by pulling in multiple sources. We could do this by using more Fetch.AI agents to scrape multiple sources for nearby events that users could attend. Similarly, we would also use Gemini's large context window to assist in that. Another thing we wanted to do was object detection in photos. So users could select specific objects in the actual photo and continue learning more with that

Outside of features, we're looking to simply run and test a wider variety of photo content so that JourneyAI can be the product it was meant to be. We've been nothing but thoroughly impressed with what we've been able to make, and we hope travelers will be too.

There's a long journey ahead for JourneyAI, but whatever comes our way, we're confident we have the tools to travel it.