Novel Sound Signal Detection

Detects novel sound signals in ambient sound in tandem with a tactile device. The absence of a novel sound will be used to confirm the existence of an auditory hallucination experienced by the user.

Comment

Sound Signal Detection Device for Verifying Auditory Hallucinations

A cognitive remediation device

Inspiration

Auditory hallucinations are a common occurrence for a larger part of the population than you might immediately think. The Cleveland Clinic writes on their page devoted to auditory hallucinations that "5% to 28% of people in the United States experience auditory hallucinations." link There is admittedly large margin in this estimation, but in my experience working as a group leader at Northwestern Medicine, many clients who have experienced auditory hallucinations find them distressing and hard to differentiate from reality. The goal of this project is to deliver a way to differentiate hallucinations from actual sounds in order to aid the lives of individuals who experience transient auditory hallucinations.

What it does

This project is a prototype device that contains a small computer, a microphone, a tactile button, and two LED lights. When the user hears something and are unsure if it is a hallucination or a genuine sound that just differs from the typical sound profile of their ambient surrounding, they are to press the tactile button which triggers the device to clip the last two minutes of the audio recording and ask Gemini if a novel sound signal is present within the last 15 seconds of the recording.

How I built it

I used a Raspberry Pi Zero 2 W running Raspberry Pi OS (64-bit) attached to an IQaudIO Codec Zero HAT to record sound and query Gemini using a Python script.

Challenges I ran into

The biggest challenge was using the correct Linux commands to properly record and clip the audio in a reasonable amount of processing time.

What I learned

I gained experience working with an IO board for the first time!

What's next for Novel Sound Signal Detection

  • Even though Gemini does a great job of differentiating signal from noise, I would like to train my own AI model for this project's specific purpose.
  • Adding a simple web app that displays the results over a course of time could be helpful for understanding the nature of the hallucinations experienced by the user-- this could be therapeutically useful information.

Built With

Share this project:

Updates