Inspiration
A few years ago, I had the privilege of meeting an incredible individual and friend.
He is a brilliant programmer and an even more remarkable person.
But what makes him truly special…is his resilience and determination in the face of adversity.
He got into an accident that left him with a severe disability, rendering him unable to move his hands and partially blind. His passion and career as a programmer were suddenly halted. It was extremely disheartening to see.
But he refused to give up. He used his technical expertise to find creative and innovative solutions to continue his work. This included VoiceOver tech and custom macros.
35% adults have 20/20 vision. ~43 million people in the world are blind. ~295 million people living with moderate-to-severe visual impairment.
VoiceOver technology is still in the dark ages.
LLMs can disrupt the field to provide VoiceOver on steroids.
What it does
Recognizant AI leverages Llama 3 and Groq enables you to understand what's on your screen without seeing. You can also speak your commands in natural language and it will respond accordingly.
How we built it
We built a frontend chrome extension that plugs into Llama 3 through Groq's inference API and combines it with what the user sees on their browser and enables an interaction model for voice to speech and speech to voice.
Challenges we ran into
- Ensuring that we actually solved a problem for the target user (i.e. people with moderate to severe vision impairment)
- Making it quick, reliable and seamless to use
Accomplishments that we're proud of
- Building something for people who need it the most
- Pushing Llama 3 to its limits
What's next for Recognizant AI
- Productionization and shipping it to the Chrome Store
- Conducting user feedback and research, iterating
Log in or sign up for Devpost to join the conversation.