HistOracle

Submission Category Age group → 18+

Submission Track → Communication 💬

Inspiration 💡

History, a crucial element in shaping one's identity, often comes off as dull and uninteresting. Understanding the importance of communicating history correctly, especially in the context of identity, motivated us to explore innovative ways to make history engaging.

Thus we came up with HistOracle! ✨

What it does 🤔

HistOracle transforms the exploration of historical figures and monuments into an interactive and delightful experience. The app caters not only to students but also to museums seeking to attract tourists worldwide. During our recent trip to Puerto Rico, we encountered the common challenge of grasping the historical significance of statues and monuments. HistOracle emerged as a solution to this problem, allowing users to interact directly with historical figures and learn about their stories.

How we built it ⚙️

We built the interface using React.js and MaterialUI. After a seamless Single Sign-On (SSO) experience, HistOracle's home dashboard appears, featuring nearby statues and historical monuments. For privacy, we derive your approximate location from your public IP, displaying relevant monuments on a Leaflet.js-powered map.

OpenAI's ChatGPT API scrapes metadata, including images, for historical sites. We've created a pipeline connecting these with preprocessed audio from Elevenlabs. Using the Audio-Visual Correlation Transformer (AVCT), we generate one-shot talking face animations, inferring motions from visual images and sample audio. Our architecture uses gRPC for high-performance APIs in microservices, with a Dockerized backend. The ChatGPT-powered chatbot works seamlessly!

Challenges we ran into 😤

With last-minute brainstorming just two hours before the hackathon and exams looming, creativity was a challenge. We opted for a minimalistic approach, emphasizing that less is more. Time constraints and the need to keep the interface simple presented additional challenges.

Communication Track 💬

In the realm of communication, HistOracle plays a pivotal role. The essence of this project lies in its ability to translate historical data into a dynamic, interactive format that speaks to a modern audience. The communication aspect of HistOracle is multifaceted, encompassing both the educational and the technological.

Interactive Learning: One of the core communication strengths of HistOracle is its capacity to turn historical learning into an interactive experience. Users aren't just passive recipients of information; they actively engage with it. This interaction makes learning more effective and memorable.
Multimedia Integration: By combining text, audio, and visual elements, HistOracle addresses various learning styles. This multimedia approach ensures that the app caters to a wide range of users, from visual learners to those who prefer auditory learning.
Cultural and Linguistic Accessibility: Recognizing the diverse user base, HistOracle is designed to be culturally and linguistically accessible. The app aims to incorporate multiple languages and cultural contexts, making history relatable and understandable to users worldwide.
User Interface Communication: The user interface of HistOracle is more than just a point of interaction; it's a communication tool in itself. The simplicity and intuitiveness of the design ensure that users of all ages and technological proficiency can navigate and benefit from the app.
Empowering Users to Explore: HistOracle empowers users to explore history at their own pace and according to their interests. This self-guided exploration is a powerful form of communication, as it allows users to form a personal connection with historical figures and events.

Accomplishments that we're proud of ✨

Despite the challenges, we successfully developed HistOracle, providing an innovative solution to the problem of making history engaging. The app's unique features, such as one-shot talking face animations and interactive conversations with historical figures, showcase our team's creativity and determination.

What we learned 🙌

Through the development of HistOracle, we learned the importance of simplicity in design, especially when faced with time constraints. The integration of various technologies, from React.js to AVCT, expanded our understanding of creating engaging and interactive applications.

What's next for HistOracle? 🚀

Beyond the hackathon, our plans include incorporating new features to cater to a wider audience. We aim to gather feedback to enhance the user interface and expand the app's reach globally.

Note ⚠️ — API credentials have been revoked. If you want to run the same on your local, use your own credentials.