Medical Robot AR

A medical robot that utilizes augmented reality technology to enable healthcare workers to perform tasks that ease stress and human labor, allowing them to have more time to be present with patients.

Comment

To be considered for:

Type 1 Future Constructor

Type 2 Interfacing with the World; Impact;

Type 3 Best in Presence

What it does

Imagine this.. you are a healthcare worker at the hospital... your shift has been extended, the inpatient ward is fully occupied, you have been tasked to check inpatient records, deliver medication, speak to inpatients ... all at once. How can we manage this hectic situation, when we only have limited staff and colleagues to assist us ? Here we introduce the TeleRobot - a remote-operated medical cart which emulates our physical presence. The robot organically responds to hand gestures and commands and performs physical tasks that we wish to complete but may be limited to do so in the hectic environments we work in.

Inspiration

Our project was developed with the UN SDGs in mind; in particular advocating for SDG #3 “Advocating for Good Health and Wellbeing”, and SDG #16 “Peace, Justice, and Strong Institutions”, which we feel are imperative in a clinical environment. Through reducing burden on essential health care workers, the telerobot promotes more equitable and sustained care in hectic work settings. We envision the telerobot being operated by the user using fluid hand gestures and voice commands (interfaced with AR/VR) to enable accessibility and precise user control. We also believe this fluid command interface captures human presence and intention in a realistic manner. On the theme of “presence”, our team sought to develop a robotic device that embodies our physical presence in circumstances where, despite our best efforts, we simply cannot be at once.

In our Tik Tok short story, we describe the scenario of “Diary of a Nurse” which recounts a busy day at the hospital. In this short film, we see nurse Veronica arriving to work, and embarking on her scheduled daily tasks - delivering medication, speaking to colleagues, clinicians, checking on her patients, and reporting patient status. She wishes she had an assistant supporting her essential role as a health care worker. She then enthusiastically hears about the telerobot; an assistive device which can help her manage the very long and complex list of responsibilities she physically engages in her workplace. Veronica and her colleagues celebrate this innovation and are hopeful it can serve an important role in hospitals of the future.

How we built it

Unity, ROS2 package (middleware) and Firmware (running on ESP32), and Assets (Blender), Arduino

Challenges we ran into

Our team members integrated hardware and software using Unity, ROS2 package (middleware) and Firmware (running on ESP32), and Assets (Blender). This presented some challenges with the ROS2 bridge. We encountered some difficulties building a robust platform to tolerate 1.5kg load. We also made 3 original Tik Tok filters which were used in our short story submission. These filters were generated using Effect House and Blender. We learned 3D modeling for the first time as well as the workflow for designing and publishing novel Tik Tok filters (wish some difficulties meshing the occluder model). As first time participants in the hardware hack track, we had the added challenge of developing physical infrastructure for the robot in parallel with our software. This required all team members to draw on their unique abilities and backgrounds in engineering and computer science.

Accomplishments that we're proud of and what we learned

This was the first time we competed in a hardware track in a hackathon environment. We found it challenging to integrate software, hardware in parallel and budget our time in both aspects. Although challenging, we were fortunate to learn from this experience as a diverse and dynamic team.

What's next for TeleRobot

The telerobot is in its early stages. Although we were enthusiastic to have an operating prototype, we realize that a number of functions still need to be embedded. To better capture physical presence, we envision the telerobot performing other clinical functions such as delivering medication/meals, verbally engaging with patients as a companionship support, generate self-reported patient status (e.g. evaluate symptoms and disposition). We also wish to merge interactivity among the health care worker, telerobot, and patient. Enabling patients to command the robot alongside healthcare workers may empower them with their care, provide peace of mind and independence.

Share this project:

Updates