Socraitive

Inspiration

Traditional testing methodologies are completely obsolete nowadays. Not only because they limit the students ability to fully express what they know, but also because AI models can now easily fool these tests. Furthermore, traditional testing methods, such as written exams and writing essays, often do not accurately measure a student's understanding or ability to apply knowledge contextually. Additionally, they may not adapt to individual learning styles or paces.

The Socratic method

Socraitive is inspired by the Socratic teaching method, which is based on fostering critical thinking through questions and dialogue. This method has been recognized for its effectiveness in deeper understanding and critical thinking. The project leverages modern AI technology to scale this method and use it to test students, providing a personalized and dynamic testing experience that adjusts to each student's responses.

What it Does

Socraitive reimagines testing by using AI to engage students in a conversational dialogue, mimicking a one-on-one interaction with a tutor. The system allows students to input study material, and in response, an AI-powered tutor generates and poses questions to probe their understanding of the material.

Fostering critical thinking through dialogue

This process not only tests knowledge but also encourages students to think critically about the subject matter. The system supports a two-way dialogue, where students can ask for clarifications, enabling a more comprehensive learning and assessment process.

Evaluation process

After the session, Socraitive evaluates the student's performance and provides a detailed grade report based on their demonstrated knowledge and engagement during the conversation.

How We Built It

Socraitive is built using Large Language Models (LLMs) like GPT-3.5 for generating and understanding conversational queries alongside with Text-to-Speech (TTS) and Speech-to-Text (STT) models to create an organic dialogue experience.

Pipeline

From the user-provided content, the system starts a conversation that includes a question. The whole communication process happens verbally.
The user will be able to speak to Socraitive and answer the question or ask a question back to it if they need more insight.
The conversation will continue happening until the model decides that it's time to grade the student.
Socraitive will then analyze the user's responses and generate a grade report based on the whole conversation.

Challenges We Ran Into

While developing Socraitive, we ran into several challenges.

One of the hardest tasks was integrating the React App frontend with the Java Spring backend. We didn't have a lot of experience with Spring, so it took us a while to figure out how to get the data sent between the frontend and backend in the correct formats, especially sending audio files.
Integrating different APIs was also difficult as we had to understand how to communicate properly with them to get the results we wanted. We used several APIs for different parts of the AI-generated response.
Another challenge was making sure the order of events was correct, so the LLM generating the response would have the correct output, so we could send it to the text-to-speech API.

Accomplishments that we are proud of

Our Minimum Viable Product looks very professional despite the timeframe we had. The website UI is easy to use and looks nice, and the backend handles all the API calls to different services. We have a very secure pipeline, as all data (voice and text) are securely sent to external APIs via their SDK.

The AI text-to-speech(TTS) is a huge accomplishment in terms of accessibility. The AI voice doesn't sound robotic, it sounds human. It also automatically switches language depending on the language spoken by the user on the first audio prompt. The TTS voice also responds to the user's questions, allowing for Socratic communication. The user can also explicitly request the App to generate a report based on their previous responses, and see how they did.

What's Next for Socraitive

We built this system with already existing AI models and APIs in less than 48 hours. With a bit more time, we have many ideas where we can expand and improve.

1. Voice Analysis

On this first version, Socraitive is only analyzing the responses from the student based on the transcript of the audio sent to the system. However, for future versions, we are planning to implement Voice Analysis, a feature that will also take into consideration the confidence, engagement, and other aspects that are unique to voice expression. This will help the system make a more accurate grading report.

2. Visual Analysis

In the current version, Socraitive does not analyze visual information such as facial expressions or drawings that could provide additional insights into a student's understanding and emotional state. In future versions, we plan to integrate Visual Analysis capabilities. This will enable the system to interpret visual cues like facial expressions or hand-drawn explanations during the test, which can offer valuable context about the student’s comprehension and engagement. By recognizing these non-verbal cues, Socraitive can provide a more nuanced assessment, similar to the feedback a human tutor might offer, enhancing the testing experience and accuracy.

3. Expand to More Teaching Domains

Currently, while effective in many subjects, Large Language Models like GPT-3.5 face challenges in subjects requiring high levels of rationalization, such as mathematics or physics, where visual aids are often crucial. To address this, we plan to enhance Socraitive with multimodal features that allow the AI to "visualize" problems in a manner similar to human students. This capability will enable the system to understand and explain complex concepts across a broader range of subjects, offering a more versatile and accurate educational tool. By incorporating tools like diagram interpretation and generation, Socraitive will better simulate a real tutor's ability to interact with and adapt to various learning materials and student needs.