KeyCatchAI

Inspiration

After learning about various cyber-attacks, and the method of side-channel attacks, we were inspired to create a tool that helps pentesters use these less common attack methods. Our attack method is inspired and follows the methodology of the paper "A Practical Deep Learning-Based Acoustic Side Channel Attack on Keyboards" by Harrison, Toreini, and Mehrnezhad (https://arxiv.org/pdf/2308.01074.pdf)

What it does

Our app records audio secretly, and, using a deep-learning ML model, transcribes the audio into a keystroke log.

How we built it

We used flutter to build the front-facing mobile app, and flask to run the backend where the ML model was running. To implement the machine learning model, we utilized libraries like pytorch, librosa, and numpy. We also developed a custom keystroke isolator that chops large audio files into separate files of single keystrokes.

Challenges we ran into

We struggled finding adequate training data for our model, and there was not enough time to develop our own robust data for training in 36 hours. We utilized a small but publicly available dataset to train our model instead. We also ran into issues processing and parsing the user recorded audio files as well as issues with background noise and different keyboards that affected the accuracy of our model. There were issues with networking the backend flask server with the mobile app, as we did not have access to a private network. We came up with the solution of using Microsoft devtunnel to securely and locally host our flask server for development.

Accomplishments that we're proud of

We successfully implemented a proof-of-concept model and app that showcases the ability for an acoustic side channel attack on keyboard audio.

What we learned

We came to understand the true difficulty of developing data and training a robust AI/ML model. It really takes a lot of training data and hours of testing to develop a truly accurate model.

What's next for KeyCatchAI

The plan for KeyCatchAI is to improve the ML model by generalizing it to many different keyboards and improving its robustness and accuracy in noisy environments.