GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in COVID-19 Classification

The project has been developed under the supervision of Prof. Paolo Russo. The work is required for the fulfilment of credits for the Vision and Perception course given by Prof. Fiora Pirri, A.Y. 2019-2020.

AUTHORS

Akash Garg, New Delhi, September 2020

Simone Rossetti, Roma, September 2020

ABSTRACT

Large-scale annotated datasets have proven to be extremely important part that powers the convolutional neural networks (CNNs) and other deep learning methods. However, it has been observed that obtaining such an enormous dataset is quite challenging in certain domains.

Medical domain is one such area where obtaining an enormous number of images in order to train a classifier is highly challenging and practically inefficient. In this project we try to address the stated critical issue with the help of Generative Adversarial Networks (GANs). We demonstrate that GAN can be used to generate synthetic medical images when trained with even small number of available real images. Not only this the quality of synthetic images is quite high and thus can be used to augment the original training dataset for further Machine Learning applications. In order to validate the quality of generated images we train a classifier firstly using classical augmentation techniques (rotation, translation, flip and scale) on smaller dataset of available real images. After that another classifier is trained but using real images, images coming from classical augmentation as well as synthetic images generated by the GAN as training dataset. In the latter case we observed an increase in sensitivity and in specificity. Adding to this the total accuracy of classification also increased significantly.

In this project we took inspiration from the work done by Maayan Frid-Adar et. al. in this paper: GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification, and by Mohamed Loey et. al. and in this paper: Within the Lack of Chest COVID-19 X-ray Dataset: A Novel Detection Model Based on GAN and Deep Transfer Learning

DATASET

COVID-19 X-rays chest scans images, 4 classes classification task:

more about dataset source here.

RESULTS:

GAN

Below are the stack of real X-ray images coming from the dataset and fake X-ray images generated from GANs trained for each of the four classes,covid, normal, pneumonia bacteria,& pneumonia virus, under consideration.

• Covid X-ray real (left) and fake (right)

• Normal X-ray real (left) and fake (right)

• Pneumonia bacteria X-ray real (left) and fake (right)

• Pneumonia virus X-ray real (left) and fake (right)

The above GAN synthetic images were generated by the models trained for 200 epochs present here. Models trained for 100 epochs here can be chosen but with a compromise with the quality of synthetic output images

CNN Classifier

We definitely achieved the high quality synthesis of chest X-ray scans using generative adversarial networks (GANs).

We designed a CNN-based solution for pneumonia infection classification task, with comparable results to state-of-the-art methods.

We demonstrated how synthetic augmentation of the CNN training set, using the generated synthetic data by GANs, highly improved classification results: the classification performance using only classic data augmentation yielded 82.7% sensitivity and 90.4% specificity. By adding the synthetic data augmentation the results increased to 90.6% sensitivity and 97.2% specificity.

Total accuracy gain, from 82.7% with classic augmented trainset to 90.7% with synthetic augmented trainset:

CNN classification performances with classic data augmentation:

CNN classification performances with synthetic (DCGAN generated) data augmentation:

Here the correspondent confusion matrix:

Training history (best - 0.907 total accuracy in 720 epochs on trainset made by 270 original images, 2000 classic and 1000 synthetic augmented):

more about CNN trained models and results achieved here.

DOCUMENTATION:

more informations about the work done here.