Turbot

What inspired us

Lukasz has just moved to San Francisco after building products in Europe, Singapore, Malaysia and Indonesia. In all of these places, the initial experience was underwhelming, with much time and effort needed to immerse himself in the surroundings and understand the historical and cultural significance of places he visited. Joined by Kelly, a Berkley CS student working on new, multicultural ways of building search and Chris, an ex-Facebook developer joining Solaris AI soon, they ventured on a mission to create a more accessible way to experience travel and learn how to build iOS apps on the way. 


What it does

Turbot uses variety of methods to understand your preferences and collect your geolocation. Then, based on the combination of the two, it uses a multiple-shot prompting technique to generate interesting messages about the location. We are currently using 250 example messages curated by us as context to generate more accurate and engaging messages for different user personas. We have audio support, for both the users communicating their preferences and the agent passing output through voice.

How we built it

We created a simple fronted, allowing users to login and collect their voice specification of preferences. We have audio specification of preferences. We then use a Node.JS backend hosted on Render to get geolocation and preferences of the users. Based on this, we use the long context window of ChatGPT-4o to compile the most relevant points and send 250 example responses for various personas to the GPT. Then, the agent performs a web search to identify new places and compile the final message. So personas and responses in the example responses are never used directly, they are just there to show GPT how to style information for various user groups (e.g. a curious child, a partygoer, a history nerd). Through this, the accuracy of taking into account user preferences and triangulating spacial points is greatly improved. We also have some settings for the users to edit and a landing page that is work in progress at tour-bot.com built with Vercel, TS and NextJS.

Challenges we ran into

Building iOS apps is difficult. Without the knowledge of Xcode and best practices for signing, team creation and sharing, we were often stopped by quirks of the software. The initial setup took until 6PM on Saturday to finally start working as intended. After that, the progress was quick, but figuring out the details and reasons for error messages was painful. 
Furthermore, the initial set of responses was very robotic and yappy. The model also had trouble understanding how far places are from each other just based on the coordinates. We had to make changes to the architecture to make the responses actually on par with what a human guide would provide.

Accomplishments that we're proud of

Building a working iOS app integrated with a variety of systems. Building a new way of utilising the long context to make spacial decisions with GPT-4o. Adding a spacial dimension to LLMs through the use of multi-shot technique, greatly improving the accuracy and style of messages being provided by the tour guide. They are interesting and varied, presenting information that is surprising and funny even for us.

What we learned

We have a much better understanding of the intricacies of combining spacial search and the web search aspect of building AI agents. We have practical experience building GPT-powered iOS apps.

What's next for Turbot

Kelly and Lukasz plan to continue working on Turbot, both in terms of finishing the app and shipping it to the App Store and potentially open sourcing it later down the road. Chris will use parts of the web search agentic features in his future startup endeavours.

Built With

• chat-gpt4o
• cursor
• swift
• typescript
• xcode