dAI-MLer

Simple Example Chat
Example Chat with Car Image

Inspiration

For most persons buying a car is a big financial commitment, especially as an electric car raises additional questions for many people. Anyone who is unable or unwilling to visit a Mercedes Benz dealer for detailed advice will have to use the online service. Unfortunately, due to the considerable number of sometimes more, sometimes less serious enquiries, not all customers can be guided through the purchase process online by Mercedes employees. This is where our chatbot can provide personalised and reliable help with the initial selection and any questions they have.

What it does

The chatbot has a friendly conversation with the potential customer in which it finds out all the attributes that are important to the buyer. In particular, it is designed to adapt to the widely varying levels of knowledge and detailed knowledge of different customers. To do this, the system analyses several data sources, aggregates the information and uses semantic searches for an output that is as context and detail-sensitive as possible. Once all the important information has been conveyed in the course of the conversation, the bot recommends a Mercedes Benz electric car that is best suited to the customer's requirements.

How we built it

Here is an overview of core technologies we used to built our RAG pipeline:

Data Collection & Preprocessing: We implemented a web crawler using Selenium in addition to the mercedes-benz-api to retrieve the data for different EQ vehicle models and their respective configurations. We further cleaned the data and filtered the relevant attributes from a consumer perspective.
Embeddings & VectorDB: After creating an object for each vehicle model, we created its OpenAI embedding. The embeddings were then added to our vector database MongoDB.
Semantic Search & Prompt Engineering: When customers send messages, our system retrieves vehicle models that match their preferences and budget by embedding the query and conducting a similarity search on our VectorDB. The extracted specifications are seamlessly integrated into the prompt engineering layer, enhancing the overall customer experience. Additionally, our chatbot dynamically adjusts its conversational style by classifying the customer's persona as the conversation progresses.
Frontend: On the frontend flutter has everything to wish for. We built a chatbot interface for our conversational LLM, which not only recommends customers the ideal vehicle for their desires and needs, but also provides them further information, images of the specific model, and links to 360° vehicle view.
Deployment: We deployed the entire backend, including our vector database MongoDB on Azure, and the frontend on Vercel: thesalesman.vercel.app

Challenges we ran into

Initiating a project of this magnitude within the given timeframe can induce feelings of overwhelm, necessitating the translation of brainstormed ideas into actionable tasks for each team member.
Conducting data crawling on the Mercedes Benz website, particularly for high-quality images, with the foresight that this additional data could enhance the chatbot in the future, presented its challenges.
Managing all data requests from the API within the constraints of a limited number of inquiries.

Accomplishments that we're proud of

Building a fully functional web app in less than 40h while having to tinker around with context-aware LLMs.
Integrating data from multiple sources, which required scraping Mercedes Benz websites of the electric vehicle fleet and utilizing their API with a very limited number of possible inquiries.
Development of a highly adaptive chatbot that can guide a potential customer through the pre-selection process regardless of their detailed knowledge of electric cars.
Being part of TUM.ai Makeathon 2024.

What we learned

Even if it goes against the intuition of a developer, sometimes it makes more sense with a tight time budget not to build the best web crawler but to keep it simple.
Prompt engineering is one of the parts with the biggest influence of the final outcome, even if it is the least ‘technical’ part, the potential should definitely not be underestimated!
Thinking about little things like a suitable group workroom with long availability can save a lot of unnecessary improvisation.
If the hot dogs run out by midnight, the sandwiches can be pimped with the remaining stuff like sauces, sour pickles, and fried onions.

What's next for The Sales Man

Further improvement of the conversation capabilities, especially with regard to more complex technical inquiries. An example of this would be inquiries about detailed usage patterns outside the usual route lengths. We are also planning to implement more extensive mechanisms to react to inappropriate inputs. With previous efforts, the chatbot just ignored every unwanted behavior. Finally, we would like to integrate the 3D Cockpit-View inside the chatbot directly without the need of links to the actual Mercedes Website.

LIVE DEPLOYMENT : thesalesman.vercel.app