Knee OA Analysis with X-ray Images, Deep Learning & Streamlit
Web app to predict knee osteoarthritis grade using Deep Learning and Streamlit
Introduction
Knee osteoarthritis is a pathology that occurs due to wear on the cartilage that protects the bones in this region from friction and impacts.
Some medical procedures are necessary to identify this pathology, such as X-rays or magnetic resonance imaging, in which it is possible to assess the loss in joint spacing, thus indicating the severity of the disease.
The severity of osteoarthritis was classified into 5 levels based on KL score, from the healthy level to the severe level, where the greater the degree of severity, the smaller the spacing of the joint.
KL score:
- Grade 0: Healthy
- Grade 1: Doubtful
- Grade 2: Minimal
- Grade 3: Moderate
- Grade 4: Severe
The following image shows the different levels from Knee Osteoarthritis Dataset with Severity Grading.
Purpose
The purpose of this project is to correctly classify the severity of osteoarthritis based on X-ray.
Methodology
The following methodology has been proposed to correctly classify the degree of osteoarthritis based on X-ray images:
- Data preparation
- Model training
- Model evaluation
- Web application development with the best model
1. Data preparation
The dataset consisting of 8000 X-ray images, approximately, of the knee obtained from the Knee Osteoarthritis Dataset.
In the bar chart we can see the image distribution of the 5 grades (classes), for each of the training, validation and test datasets, and in the pie chart we can see the average percentage of data we have for each class. So we have an unbalanced dataset.
Three strategies were implemented to reduce the impact that the unbalanced base can have on the models:
- class weight
- data augmentation (horizontal_flip, brightness_range, width_shift_range, zoom_range)
- preprocessing features of pre-trained networks
See 01_data_preparation.ipynb notebook.
2. Model training
Pre-trained Networks
Three pre-trained networks were chosen: Xception, ResNet-50 e Inception Resnet v2.
The following table presents the summary of the partial results obtained in the implementation of the different pre-trained networks with fine tuning. Our metrics is Balanced Accuracy. Models were trained on Apple M1 Pro chip with 8-core CPU, 14-core GPU and 16-core Neural Engine.
Highlighting, the highest success rate of each model by class, we have:
- Inception ResNet classified the minimal class better
- Xception classified the doubtful and severe classes better
- ResNet50 classified the healthy and moderate classes better
Ensemble
Three ensemble approaches were performed with the previous results:
- One calculating the average, and two calculating the weighted average of the accuracy and f1 of each class.
The three models had similar results, but we selected the ensemble with f1 model.
Model evaluation
We evaluated the best model in the test set, a balanced accuracy of 71% was obtained, and in the confusion matrix we can observe the highlight of the moderate and severe classes.
Explainability
We implemented the Grad-CAM explainability technique to better understand how classes are classified. The Grad-CAM indicates the parts of the image that most impact the classification score.
We can see in the images that for the healthy, doubtful and minimal classes, the most prominent areas are located in the center of the knee, and the moderate and severe classes are most prominent on the right or left edges of the knee.
Grad-CAM results were obtained from the last convolutional layer of the Xception model.
Web application development with the best model
The web application allows you to select and load an X-Ray image, to later predict and evaluate the loss in joint spacing, and indicate the probability of disease severity, as well as the area that most impacted the classification score.
