snowTok

• 

  snowTok - streamlit web application

Inspiration

The inspiration behind snowTok came from the desire to make video creation more accessible and efficient. With the rise of short-form video content on platforms like TikTok, there was a clear need for a tool that could simplify the process of creating engaging videos. By leveraging the power of artificial intelligence, snowTok aims to empower users to bring their creative ideas to life without the need for extensive editing skills or resources.

What it does

snowTok is a web-based application that generates TikTok-style videos from simple text prompts. Users can input a description of the video they want to create, and snowTok utilizes various AI models to generate visually appealing images, engaging audio, and synchronize them into a cohesive video. The resulting video is polished and ready to be shared on social media platforms.

How we built it

snowTok was built using a combination of AI models and web technologies. The frontend of the application was developed using Streamlit, a Python library for building interactive web apps. The backend leverages several AI models, including the Snowflake Arctic LLM for generating image descriptions and text-to-speech content, the fast-lightning-sdxl model for image generation, OpenAI's TTS-1 model for audio creation, and Deepgram's nova-2 model for audio transcription. These models work together to transform the user's text prompt into a complete video.

Challenges we ran into

One of the significant challenges faced during the development of snowTok was optimizing the video rendering process. Initially, the rendering of videos took a considerable amount of time, often exceeding 3 minutes per video. This posed a major hurdle in providing a seamless user experience and efficiently handling user requests. To address this issue, I implemented parallel execution techniques utilizing GPU resources, the rendering time was substantially reduced to less than a minute, resulting in a much faster and more responsive application.

Accomplishments that we're proud of

One of the major accomplishments of snowTok is its ability to simplify the video creation process and make it accessible to a wider audience. By leveraging AI technologies, snowTok eliminates the need for complex editing tools and skills, allowing users to focus on their creativity. The seamless integration of multiple AI models to generate high-quality videos is another proud achievement of the project.

What we learned

One of the key learnings was the importance of integrating various AI models effectively. The project involved working with different models, including Snowflake Arctic, which provided structured outputs for generating image descriptions and text-to-speech content. Integrating these models seamlessly posed challenges but also offered opportunities to leverage their strengths and create a cohesive video generation pipeline.

What's next for snowTok

The future of snowTok is filled with exciting possibilities and potential enhancements. One of the key areas of focus will be expanding the customization options available to users. This includes allowing users to input their own images and custom music tracks to create personalized reels and videos

Built With

• amazon-web-services
• deepgram
• fal-ai
• fastapi
• modal-labs
• openai
• replicate
• s3
• snowflake
• stablediffusion
• streamlit
• upstash