-
-
Fig. 1: Map of Covid19 cases around the world (as of 4/30/2020)
-
Fig 2: Current chest X-ray diagnosis vs. noval process with CovidScan.ai
-
Fig. 3: Chart of wait-time reduction of AI radiology tool (data from a simulation stud reported in Mauro et al., 2019).
-
Fig. 4: Process of CovidScan development
Inspiration
What will be working situation for medical staff in hospitals during and after the COVID-19 pandemic? How can the medical staff quickly and securely log in and perform PPE safety check while dealing with a huge influx of patients in critical conditions? How can we automate the process of COVID-19 diagnosis so precious time can be saved for both medical doctors and the patients? How can our solution for hospital later be scaled and implemented to be a essential tool for automating the daily operation at hospital even after the COVID-19 pandemics is over?
To answer these core questions, we did some background research to identify the main challenges in order to develop the best solutions around those:
COVID-19 Pandemic:
Fig. 1: Map of Covid19 cases around the world (as of 4/30/2020). Our team created the map based on data collected by the Johns Hopkins University Center for Systems Science and Engineering.
As we see from the map above and the pie chart below, COVID-19, previously known as the novel Coronavirus, has killed more than 63,860 people and infected over 1,067,061 people in the United States alone, topping all other countries around the world. This number is continuing to grow every day.
Fig. 2: Top 10 countries with most COVID-19 deaths.
The main problem occur in the healthcare system during the pandemics is the long wait time for COVID-19 chest X-ray result:**
Fig 3: Current chest X-ray diagnosis vs. novel process with CovidScan.ai
Patients can first be screened for flu-like symptoms using nasal swap to confirm their COVID-19 status. After 14 days of quarantine for confirmed cases, the hospital draws the patient’s blood and takes the patient’s chest X-ray. Chest X-ray is a golden standard for physicians and radiologists to check for the infection caused by the virus. An x-ray imaging will allow your doctor to see your lungs, heart and blood vessels to help determine if you have pneumonia. When interpreting the x-ray, the radiologist will look for white spots in the lungs (called infiltrates) that identify an infection. This exam, together with other vital signs such as temperature, or flu-like symptoms, will also help doctors determine whether a patient is infected with COVID-19 or other pneumonia-related diseases. The standard procedure of pneumonia diagnosis involves a radiologist reviewing chest x-ray images and send the result report to a patient’s primary care physician (PCP), who then will discuss the results with the patient.
_Fig 4: Chart of wait-time reduction of AI radiology tool (data from a simulation stud reported in Mauro et al., 2019). _
A survey by the University of Michigan shows that patients usually expect the result came back after 2-3 days a chest X-ray test for pneumonia. (Crist, 2017) However, the average wait time for the patients is 11 days (2 weeks). This long delay happens because radiologists usually need at least 20 minutes to review the X-ray while the number of images keeps stacking up after each operation day of the clinic. New research has found that an artificial intelligence (AI) radiology platform such as our CovidScan.ai can dramatically reduce the patient’s wait time significantly, cutting the average delay from 11 days to less than 3 days for abnormal radiographs with critical findings. (Mauro et al., 2019) With this wait-tine reduction, patients I critical cases will receive their results faster, and receive appropriate care sooner.
What it does
Using the power of pretrained machine learning models from open source, CovidScan.ai is created as a full-scaled AI tool for radiology clinics and hospitals. It can automate the process of detecting sign of COVID-19 and pneumonia on chest X-ray images to assist radiologists during the pandemics. This tool of cutting edge technology can be used to reduce the workload for clinicians, and speed up patients’ wait time for pneumonia lab results in this critical time of the COVID-19 pandemic.
In summary, a patient who need COVID-19 testing will go through the following process using our application:
- A user answers a series of questions using an algorithm built to identify whether they need additional screening or not.
- If they need additional screening/X-Ray then we proceed to use their postal code to geo-locate the nearest hospitals with testing available
- Once the case reaches at that point, the user just waits and it advances to a physician’s worklist The physician opens the case and looks through the information and uploads X-ray images to identify whether the patient tests positive for pneumonia
- The process of sending out for an X-Ray and getting them back are excluded from this application. The X-Rays could also be part of the patient’s existing medical records which could easily be located by the hospital’s system
Benefit of COVIDSCAN APP:
Using this application, the medical staff take patients’ chest X-ray images using the specialized machine and then upload the taken images to the database of web-app for testing for sign of COVID-19 infection or bacterial pneumonia. It is due to the fact that an AI system can review, highlight the pneumonia sign and classify each X-ray image all in less than 10 seconds (comparing the radiologist’s 20 minutes that we mentioned earlier), and it can do that same task effortlessly for 24 hours without taking a break. This time cut is especially critical in the time amid the pandemic of COVID-19. With this spreading rate, it will be overwhelming for radiologists to review a massive number of chest X-ray images of potential COVID-19 infected patients. With the assistance of CovidScan.ai, it can automatically highlight the suspected signs of pneumonia for the radiologists and speed up the process of chest X-ray review. Therefore, more COVID-19 positive-tested patients will get their result back faster and receive appropriate care sooner to prevent the spread of the virus.
How we built it
Pega intergration:
We create a functional solution that incorporates the chest X-ray image detection model into Pega workflow. Platform used is Pega 8.2 and the application is scalable to different diagnosis and future disease recognition as CovidScan expands. Workflow was designed using Pega’s out of the box App creation and case management without any customization. The integration to the REST API’ was done by using the pega connectors. Due to the limited time, we have simulated the input and output for the hackathon.
Chest X-ray Classification Model: For the deep learning model, we developed a Pytorch model. This project’s goal is to draw class activation heatmaps on suspected signs of pneumonia and then classify chest x-ray images as “Pneumonia” or “Normal”. For this project, we are going to use a dataset available at Kaggle consisting of 5433 training data points, 624 validation data points and 16 test data points. C. For the model, we load the pre-trained Resnet-152 available from Torchvision for transfer learning. ResNet-152 provides the state-of-art feature extraction since it is trained on a big dataset of ImageNet. ResNet-152, as the name sounds, consists of 152 convolutional layers. Due to its very deep network, the layers are arranged in a series of Residual blocks. These Residual blocks skip connections to help prevent the vanishing gradients which are a common problem with networks with deep architecture like ours. Resnet also supports Global Average Pooling Layer which is essential for our attention layer later on. For the attention layer to draw the heatmap, we use the global average pooling layer proposed in Zhou et al. Global average pooling layer explicitly enables the convolutional neural network (CNN) to have remarkable localization ability. We achieve 97% accuracy on the training dataset and 80% on the testing dataset.
Technical Requirements:
The packages required for this project are as follows:
Pega 8.3
Pega workflow
Torch (torch.nn, torch.optim, torchvision, torchvision.transforms)
Django
Numpy
Matplotlib
Scipy
PIL
Tensorflow
jQuery
Challenges we ran into
This hackathon project was a very different experience for us which challenged us throughout this project in the Pega intergration and deep learning model training part. This is the first time we all were working with creating endpoints of the pre-trained deep learning model to intergrate to Pega workflow.
Accomplishments that we're proud of
We manage to finish the project in such a limited time of 2 weeks in our free time from school and work. We still keep striving to submit on time while learning and developing at the same time. We are really satisfied and proud of our final product for the hackathon.
What we learned
Through this project, we learn to deploy a complicated image-recognition deep learning models on Pega platform. We also learn the process of developing a mini data science project from finding dataset to training the deep learning model and finally deploy & integrate it into a web-app. This project can’t be done without the efforts and collaboration from a team with such diverse backgrounds in technical skills.
What's next for CovidScan:
In the next 2 months, our plan is:
We will raise fund to invest more into the R&D process.
We will partner with research lab to collect more dataset and find hospitals to test our solution. One of our memeber has published his newly collected dataset on this open-source github: https://github.com/nihalnihalani/COVID19-Detection-using-X-ray-images-/
Regarding our R&D, we plan on improving the performance of the platform, preferably by reading more scientific literature on state-of-art deep learning models implemented for radiology.
We also plan to add the bound box around the suspected area of infection on top of the heatmap to make the output image more interpretable for the radiologists. We are working to implament the multilabeling model of COVID-CXR on our dataset to improve our application. This model is published by The Artificial Intelligence Research and Innovation Lab at the City of London's Information Technology Services division and has accuracy 0.92, precision 0.5, recall 0.875, auc 0.96.
In many pieces of literature, they mentioned developing the NLP model on radiology report with other structured variables such as age, race, gender, temperature... and integrating it with the computer vision model of chest X-ray to give the expert radiologist’s level of diagnosis. (Irvin et al., 2019; Mauro et al., 2019) We may try to implement that as we move further with the project in the future.
With the improved results, we will publish these findings and methodologies in a user-interface journal so that it can be reviewed by expert computer scientists and radiologists in the field.
Eventually, we will expand our classes to include more pneumonia-related diseases such as atelectasis, cardiomegaly, effusion, infiltration, etc. so that this platform can be widely used by the radiologists for general diagnosis even after the COVID-19 pandemics is over. Our end goal is to make this tool a scalable that can be used in all the radiology clinic across the globe, even in the rural area with limited access to the internet like those in Southeast Asia or Africa.
References:
Crist, C. (2017, November 30). Radiologists want patients to get test results faster. Retrieved from https://www.reuters.com/article/us-radiology-results-timeliness/radiologists-want-patients-to-get-test-results-faster-idUSKBN1DH2R6
Irvin, Jeremy & Rajpurkar, Pranav & Ko, Michael & Yu, Yifan & Ciurea-Ilcus, Silviana & Chute, Chris & Marklund, Henrik & Haghgoo, Behzad & Ball, Robyn & Shpanskaya, Katie & Seekins, Jayne & Mong, David & Halabi, Safwan & Sandberg, Jesse & Jones, Ricky & Larson, David & Langlotz, Curtis & Patel, Bhavik & Lungren, Matthew & Ng, Andrew. (2019). CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison.
Kent, J. (2019, September 30). Artificial Intelligence System Analyzes Chest X-Rays in 10 Seconds. Retrieved from https://healthitanalytics.com/news/artificial-intelligence-system-analyzes-chest-x-rays-in-10-seconds Lambert, J. (2020, March 11). What WHO calling the coronavirus outbreak a pandemic means. Retrieved from https://www.sciencenews.org/article/coronavirus-outbreak-who-pandemic
Mauro Annarumma, Samuel J. Withey, Robert J. Bakewell, Emanuele Pesce, Vicky Goh, Giovanni Montana. (2019). Automated Triaging of Adult Chest Radiographs with Deep Artificial Neural Networks. Radiology; 180921 DOI: 10.1148/radiol.2018180921
Wang, L., & Wong, A. (2020, March 30). COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest Radiography Images. Retrieved from https://arxiv.org/abs/2003.09871
Log in or sign up for Devpost to join the conversation.