logo

Learn To Live & Live To Learn

Inspiration

I have always been inspired by AI and Innovation. I have learned a lot through YouTube video tutorials. YouTube is the very next go to favorite teacher of every person in the face of earth. The feature which is not available with this teacher is the question and answer feature. Now with the availability of Gemini API key and generative content creation YouTube teacher can be enabled.

What it does

A simple extension which can be used through the chrome side panel. First the user will provide the you tube video link and then a chat session will be created. In the beginning the chat will generate short and crisp transcript of the video with timestamp. Later this will answer to question which user will ask.

How we built it

First we did a market study and understood thoroughly how current AI based products are performing. We all suggested our top ideas among them you tube video related ideas were encouraged a lot. So we started out very ambitiously with so many deliverables. we all used to regularly connect and discuss a lot. We have developed so so many features but due to time constraint we were unable to execute many things. I started my work by analyzing the you tube content videos with pytube python library. I also studied about ffmpeg library. I understood about the audio formats and the size associated with audio tracks.

Due to time constraint I had to combine my efforts and start working so that i can deliver a MVP in this hackathon.

Arachnospec now is able to help users to understand you tube very accurately and precisely.

Stack used: Frontend: Chrome extension showing react based application running at port 3000 with the help of iframe. backend: flask server running at port 5000 middleware api: gemini latest model api calls.

Methodology: Audio files converted to relevant format in minimum sampling rate and bitrate. multi chanel to mono channel conversion. denoising and speach audio retrieval. creatin chat response base on the audio file uploaded.

Challenges we ran into

Understanding the api key usage was a problem. I even ended up using whole google cloud platform. I even have some bill amount to pay : ) . First we thought GCP platform can be used but later understood the credits are only for gemini ai api.

With respect to project development the api calls frequency quota was limited. It was hard to maintain a quick chat response. To manage lengthy video we need to use chunk based processing with concurency.

Accomplishments that we're proud of

Its my first hackathon and I am really proud of the fact that i got a team and we connected on a regular basis and brainstormed a lot. I learnt a lot on a very global scale.

What we learned

Google Cloud platform workspace. Compute. I ended up even having an active domain now Arachnospec (fan of spider man ). Gemini google ai studio google vertex audio file processing Chat bot Chrome extension

What's next for Arachnospec

There is a lot in bucket actually. We have gone through a lot of ai solutions and cannot wait to continuously developing the product. LOL Thanks to this hackathon i have got a domain now arachnospec.com

Github repository creation from youtube video coding tutorials. Video based dataset creation for ML models. Content based chat widgets and icons with styles. Multimodal search with audio and video samples across video.