IRL - Mixed-Reality For The Masses

Expand your vision with IRL–merging Infrared Light into real life through VR, Kinect V2 data, and real-time object detection. Enhance safety, connect activities, and extend your night vision spectrum.

Comment 1

Inspiration

The inspiration behind IRL was sparked by the emergence of new mixed reality headsets, such as Apple Vision Pro and Meta Quest 3. Witnessing the potential of these advanced technologies motivated us to bridge the gap and bring a level of mixed reality to conventional VR headsets. We aimed to leverage this innovation to enhance safety in low-light situations during night activities like walking, biking, and driving.

What it does

IRL utilizes the Xbox One Kinect V2 Sensor with libfreenect2 to extract crucial data such as IR, 1080p camera, and depth sensing. The system transforms this data into a VR split-screen view, allowing users to wear VR headsets and navigate the night with enhanced visibility. YoloV5 adds a layer of safety by providing real-time object detection, identifying potential obstacles and hazards.

How we built it

Combining the power of libfreenect2 from GitHub for Kinect V2 data extraction, integrating OBS for VR split-screen visualization, and incorporating YoloV5 for robust object detection, we strived to bring a mixed reality experience to conventional VR headsets. The project also involves streaming the experience on platforms like YouTube, Twitch, and Zoom for real-time sharing of safety information.

Challenges we ran into

Extracting data from the discontinued Kinect V2 proved challenging due to limited support, but our determination led to a successful integration. Converting the data into a VR-accessible format and establishing a smooth live streaming process required careful consideration of compatibility and optimization.

Accomplishments that we're proud of

Amidst challenges, we successfully created a cost-effective solution that combines VR and Kinect V2 data for enhanced safety. This project marked our inaugural foray into VR development, showcasing our ability to adapt and learn rapidly in a cutting-edge tech space.

What we learned

The project illuminated the intricacies of working with discontinued hardware and introduced us to the exciting realm of VR development. Learning how to merge mixed reality concepts into conventional VR headsets opened doors to new possibilities and creative problem-solving.

What's next for IRL

Looking forward, our goal is to create a more portable version, resembling current production mixed reality headsets. We aim to transform the system into a wearable format, making use of the IR sensor and other sensors for enhanced safety in a compact, user-friendly design. Inspired by the success and learnings of IRL, we plan to continue pushing the boundaries of innovation in safety technology.

Built With

Share this project:

Updates