SearchableVideos

Inspiration

Myself and my team mates thought we were missing out on various knowledge shares or meetings that were recorded within our company. We also found it very hard to find the recordings that contained relevant knowledge to us. I discussed it with them and came up with an idea to have these videos transcribed and posted to slack so we could search for them there. Slack is where we usually search for knowledge within our company.

What it does

SearchableVideos allows a user to choose a video recording from their google drive and a slack channel. The app then transcribes the video and posts a google drive link to the chosen slack channel with the transcript posted in the thread.

How I built it

I used the Google picker API widget to allow the user to log into their google drive and choose the video they would like to transcribe. For the slack app I used the bolt js framework to handle all the events that happened in slack, from google drive buttons, to using modals views when choosing a slack channel. Along with using the slack app bot to post the video link and transcript to the chosen slack channel.

I downloaded the video file through Google drive API from the fileId and access token captured in the Google Picker Widget API. Googles speech to text service was used to transcribe the video file. To do this I needed to transform the video mp4 file to an audio flac file so this service could handle it. After that it was a matter of creating time stamp messages from the transcript so the slack app bot could post the them to the relevant slack thread.

Challenges I ran into

It was difficult to route the user to the Google Picker widget, store the access token from the OAuth process and then redirect the user back to the slack app. Since I am not that experienced with javascript, I found this difficult.

Gathering the timestamp data from the transcript created by Googles speech to text service was difficult and involved combing through api documentation to find the correct fields to use. After that it was important to perform some operations synchronously like posting the transcript messages to the slack post thread.

Accomplishments that I'm proud of

Successfully transcribing video recordings and posting the transcript to a chosen slack channel. The videos are now searchable. Implementing the Google Picker widget and redirecting the user back to the slack application. Using the bolt framework to send the transcript timestamp messages to the slack post thread which contained the video url. Using the Google speech to text service to transcribe audio files was pretty great.

What I learned

It is important to think how a user will interact with your application early on. Simplify your idea to have a working end to end product and iterate from there. Think about how authentication is going to interact with your user flow.

Be careful in how you understand api documentation. Having the correct understanding of api fields is crucial.

What's next for SearchableVideos

I would like to implement google buckets to store the audio files associated with the video recording so it can handle very large recordings. Implement better persistent storage for the video files and user information. Allow the user to select multiple recordings at once to transcribe. Have the timestamps in the transcript link to the play time video urls would be pretty cool.