Advanced Detection of Retinal Diseases via Novel Hybrid Deep Learning Approach

The human eye, a vital sensory organ, significantly influences our quality of life. Unfortunately, today, retinal diseases pose a severe threat, often leading to substantial visual impairment and even blindness. Among the most common retinal diseases are Myopia, Drusen, Diabetic Retinopathy (DR), Diabetic Hypertension (DH), Diabetic Macular Edema (DME), Glaucoma, Choroidal Neovascularization (CNV), Cataract, and Age-related Macular Degeneration (AMD). Early detection of these diseases is paramount to prevent irreversible ocular disorders [1]. In this paper, we present a study focusing on the detection of CNV, DME, and Drusen, which are among the most prevalent retinal diseases [2-10] and extensively discussed in the literature [11-20].

CNV is characterized by the formation of small, irregular blood vessels in the retina [21]. The inability to transport oxygen through the retinal blood vessels often leads to their abnormal dilation and fragility [22]. The resulting delicate vessels rupture under minimal pressure, resulting in retinal hemorrhages. CNV prompts visual loss due to vitreous hemorrhage and vasodilation. More than 200,000 people in the United States are diagnosed with CNV each year [23]. DME, on the other hand, is a condition prevalent among diabetics, caused by high blood sugar levels damaging the retinal vessels. The fluid from these compromised vessels leaks into the macula, leading to blurred vision [24]. Drusen comprises yellow deposits of lipids located beneath the retina [25].

The progression and detection of retinal diseases are observed through irregularities in or near various parts of the eye's retina, including the optic disc, blood vessels, and fovea [26]. Retinal images are essential for these observations. To accurately analyze retinal images, structures like the optic disc and macula in the retina's natural structure must be correctly identified and differentiated from degenerative signs [27]. Ophthalmologists diagnose retinal diseases by analyzing these images. However, some studies predict that in the future, a scarcity of ophthalmologists may render them unable to cater to all retinal patients [28]. This stark disproportion between retinal patients and the number of ophthalmologists could lead to service delays. Hence, in such a scenario, there would be a pressing need for systems that can automatically detect eye diseases, thereby assisting ophthalmologists in managing their workload.

Optical Coherence Tomography (OCT) is an imaging technique capable of evaluating retinal structures at microscopic resolution [29]. OCT measures the time delay between light emission and its reflection from the target tissue, akin to the sound used in ultrasonography. Due to the high speed of light scans, the reflection is measured as direct detection is unfeasible. OCT scans are obtained by rapidly scanning the macula from all sides with an OCT light source. Similar to ultrasound images, the B-scan is combined to form a linear image [30]. Currently, only a limited number of datasets containing OCT images are available. In this study, we utilized the publicly accessible OCT image dataset presented by Kermany et al. [31] in 2018 to detect the aforementioned retinal diseases.

In recent years, the medical field has increasingly leveraged artificial intelligence (AI) methodologies, with deep learning techniques being applied to medical imaging analysis to yield significant advancements. The literature review section of this study highlights the numerous recent investigations that utilize deep learning models for the analysis of retinal images.

The primary objective of this study is to employ a deep learning approach to detect retinal diseases with a high degree of accuracy. To accomplish this, we first applied nine different deep learning models to the OCT image database and analyzed their disease detection success rates. Additionally, we performed hyperparameter tuning to enhance these models' performance. Subsequently, we developed a hybrid model that we term EffXCeptNet. This model combines two pre-trained convolutional neural networks (CNNs), namely EfficientNetV2S and Xception, with modified hyperparameters. To demonstrate the efficacy of our proposed model, we conducted performance evaluations and presented the accuracy percentages of the discussed models. According to these evaluations, the proposed deep learning model, EffXceptNet, achieved accuracy, recall, and precision values of 99.90%. The performance analysis results indicate that EffXCeptNet outperforms the nine other deep learning models applied for the considered eye diseases. While the superior performance of EffXCeptNet is demonstrated for the eye diseases discussed in this study, we expect this model to yield equally successful results when trained with different medical image sets, implying that it can be applied for the detection of various diseases.

This study is organized into five sections:

The Introduction section provides an overview of retinal diseases, deep learning methodology, and the aim of this study.

The Literature Review section summarizes previous studies on the detection of retinal diseases.

The Material and Methods section describes the dataset used in this study for retinal disease classification, the deep learning methods employed, and the proposed hybrid deep learning model.

The Results section presents the performance outcomes of the examined models. It also demonstrates the efficiency and success of the proposed model by providing comparative analyses of the performance results of the discussed models and the proposed model.

The Conclusion section summarizes the key findings of the study.

In this study, different versions of CNN architecture were applied to OCT retinal images database to detect CNV, DME and Drusen patients and the performance results of these models were obtained. The CNN models discussed in the study are RestNet50, InceptionV3, InceptionResnetV2, MobileNet, DenseNet-201, Xception, EfficentNetB4, EfficentNetB7 and EfficentNetV2S. After that, hyperparameter tuning process was performed for each of these models to increase their classification performance. Then, the proposed model, which is a hybrid version of the most successful of these hyperparameter tuned models, EfficientNetV2S and Xception CNN models, was developed.

The study can be evaluated in 4 steps:

1. We trained discussed CNN models (RestNet50, InceptionV3, InceptionResnetV2, MobileNet, DenseNet-201, Xception, EfficentNetB4, EfficentNetB7, EfficentNetV2S) applying OCT image dataset and obtained the performance results of each.
2. We performed hyperparameter tuning to these CNN models to increase the classification performances of them and presented new performance results.
3. We developed hybrid CNN model based on the most successful models of these hyperparameter tuned CNN architectures and measured performance results of our model.
4. We showed the efficiency and success of the proposed model by representing comparisons of the performance results of the discussed models and the proposed model.

3.1 Convolutional neural network (CNN)

CNN architecture is a deep learning model often used for classification in applications with many data, such as images.

CNN is made up of neurons with biases and weights that need to be trained in a manner similar to conventional neural networks. Multiplication of inputs and weights is applied in each neuron [46]. CNN architecture is a combination of a series of successive layers. Following the input layer, where an image with width and height is used, as shown in Figure 1, there are three main layers: fully connected, pooling, and convolution. These layers are added one after the other to form a CNN architecture [47].

3.1.1 Convolutional layer

The convolutional layer detects features of images such as lines, edges, drops of color, etc. The convolutional layer creates a feature map that applies a filter to all pixels of the image at each cycle and estimates class probabilities based on image features. The filter acts on the parts that make up the image to check whether the desired feature is available [48-50].

3.1.2 Pooling layer

The pooling or subsampling layer is used to reduce the width and height of the activation map, as shown in Figure 2. They are special pooling types where the average and maximum values are taken. In general, they reduce the number of parameters and calculations that increase due to the increase in the number of convolutional layers in the network [51-53].

3.1.3 Fully connected layer

Fully connected layers come after pooling layers. Attributes are obtained from the outputs of the pooling and convolution layers from the input images. The fully connected layer uses these properties to classify the image into a predetermined class.

3.2 The CNN models discussed

3.2.1 ResNET50 network

He et al. [54] used a network structure called residual neural network (ResNet), which took first place at the ImageNet Large Scale Visual Recognition Challenge (ILSVRC 2015). ResNet proposed residual connections between layers, which help preserve knowledge gain, reduce loss, and boost performance during the training. ResNet50 consists of 50 layers, including 48 convolution layers, 1 Average Pooling layer, and 1 Max Pooling layer. Figure 3 shows the structure of the ResNet architecture.

1.png

Figure 1. Layers of convolutional neural networks

2.png

Figure 2. Max pooling and average pooling filter

3.2.2 InceptionV3 network

The Inception network is a pre-trained CNN model introduced by Google in 2014 [55]. This network consists of 22 layers, with max pooling and filters of different sizes used to extract features at various scales. Small filters are used to save time in calculations. In 2015, Google introduced 48-layer InceptionV3 [56], where Convolution layers are factored to reduce parameters in the Inception model. Two 3×3 filters were used instead of 5×5 convolutional filters to reduce computation without affecting the performance of the networks. The structure of the InceptionV3 architecture is shown in Figure 4.

3.2.3 InceptionResnetV2 network

InceptionResNetV2 is a network consisting of residual connections [54] and a combination of the Inception architecture [56]. Improves network performance while avoiding gradient disappearance and gradient explosion [57]. The block diagram of the InceptionResnetV2 architecture is shown in Figure 5.

3.png

Figure 3. The architecture of the ResNET50 network model

4.png

Figure 4. Architecture of InceptionV3 network model

5.png

Figure 5. Architecture of InceptionResnetV2 network model

3.2.4 MobileNet network

MobileNet consists of light deep CNN using deeply separable convolutions. This architecture provides an efficient model for embedded systems and mobile applications [58]. Deeply separable convolution filters consist of point convolution filters and deep convolution filters. MobileNet's Deeply separable convolution module compresses both computational complexity and parameters. It accelerates image recognition without loss of accuracy as it enables embedded and mobile devices to take full advantage of the processing power of the GPU and CPU [59]. Figure 6 shows the structure of the MobileNet architecture.

3.2.5 DenseNet-201 network

The DenseNet model consists of transition layers, dense blocks, growth rates, and the global average pooling layer [60]. While In K-layer convolutional networks, each layer has a K connection with the next layer, the DenseNet model has a K(K+1)/2 connection. Feature maps of each layer and feature maps of all previous layers are used as inputs for all subsequent layers [61]. DenseNet-201 has three transition layers and four dense block depths. Each dense block has two convolution layers of different sizes. Each convolution layer consists of rectified linear units and batch normalization [60]. DenseNet has many advantages, such as strengthening feature propagation, promoting feature reuse, reducing the vanishing gradient problem, and reducing the number of parameters [62]. The structure of the DenseNet-201 architecture is shown in Figure 7.

3.2.6 Xception network

The Xception architecture [63] introduced by Francois Chollet is an enhanced version of InceptionV3 [56]. The Xception model is based on deeply separable convolution, which divides ordinary convolution into spatial convolution and point convolution. Spatial convolution occurs independently on each input channel. Point-to-point convolution uses 1×1 kernel to evolve from point to point. As the number of parameters decreases, the number of calculations also decreases. Xception consists of 14 modules with 36 convolution layers. Except for the first and last modules, the other modules contain linear residual connections. The structure of the Xception model architecture is shown in Figure 8.

3.2.7 EfficientNet Network

Tan and Le [64] proposed EfficientNet model, which shows higher sensitivity and is more flexible than CNNs. The authors used efficient and simple composite coefficients to uniformly scale the EfficientNet models depth, width, and resolution dimensions. This enables the network using a fixed resource budget to achieve higher performance. The EfficientNet family consists of eight CNN models, named from EfficientNetB0 to EfficientNetB7. From EfficientNetB0 to EfficientNetB7, resolution, width, depth, model size, and accuracy are increasing.

6.png

Figure 6. Architecture of MobileNet network model

7.png

Figure 7. Architecture of DenseNet-201 network model

8.png

Figure 8. Architecture of Xception network model

9.png

Figure 9. a) is an example of a basic network. b-d) are conventional scaling that only increases one network width, depth, or resolution dimension. e) is the recommended composite scaling method that evenly scales all three dimensions with a fixed ratio [64]

The EfficientNet architecture consists of excitation and squeeze optimization and Mobile Inverted Bottleneck Convolution (MBConv) [65]. MBConv expands the channel by 1×1 convolution method, then each image performs a deep convolution operation. Deep convolution becomes a feature map by performing a convolution method with d x d cores for all image channels. Each layer goes through the Swish function for the enable function. Unlike the Swish, Tanh and Sigmoid functions, it prevents the gradient value from reaching near zero saturation throughout the learning process. The extracted feature map is replicated with a feature map that bypasses the Compression and Excitation Layer to reveal key features. Finally, the channel is scaled down by a 1×1 convolution operation.

EfficientNetV2 recommended by Tan and Le [66], has higher performance and a shorter training time. It uses the Fused MBConv layer instead of some MBConv layers used in EfficientNet to increase training and efficiency and is up to 6.8x faster than current models. The EfficientNetV2 architecture is available in three versions, EfficientNetV2S, EfficientNetV2M, and EfficientNetV2L.

Figure 9 shows the EfficientNet scaling model. Figure 9 (a)–(d) shows a basic mesh and a traditional scaling technique, respectively, while Figure 9 (e) shows the composite scaling introduced by Tan and Le [64].

3.3 Proposed hybrid CNN model – EffXceptNet

To detect retinal diseases, we first applied a deep learning model with nine different hyperparameter settings in the OCT image database. Next, we developed a hybrid model combining the most successful of these models, EfficientNetV2S and Xception. Figure 10 shows the development stages of our model.

Figure 11 shows the developed hybrid deep learning model architecture. It consists of the combination of two CNN models which are EfficientNetV2S and Xception, two flatten layers, three fully connected layers, and an output layer. An image of 150×150 pixels is given to the model in the input layer. In our proposed model, two CNN models with high performance on the OCT dataset were combined. The Xception model produces 2,048 features in training, and the EfficientNetV2S model produces 1,280 features in training. A total of 3.328 features are obtained by combining the features of both models with the concatenate method. These features are given as input during the classification phase. ReLU activation function is used in fully connected layers. At the same time, Lasso Regression (L1 regularization) value was chosen as 0.001 in fully connected layers to prevent overfitting. The softmax function used in the last layer of the proposed architecture calculates the probabilistic distribution of the input image for four classes in the trained network.

10.png

Figure 10. The development stages of the proposed hybrid model

11.png

Figure 11. Structure of proposed hybrid model-EffXceptNet

The model we propose is trained with the Adam optimizer. The Adam optimization algorithm has recently been widely used in deep learning applications in computer vision. There are various studies in the literature about the efficiency of Adam Optimizer and its effective results in deep learning [67-70]. In our study, the Adam optimization algorithm was applied to improve the weight coefficients of the network.

Learning rate is one of the most important hyper parameters in model training. We have choose learning rate as 0,001 initially, after several trainings on the training and validation data sets. Learning rates were chosen as 0.1, 0.01, 0.001 and 0.0001, respectively. The best performance and ideal training speed were achieved in model training, where the learning rate was initially selected as 0.001. During the model training, we halved the learning rate in each epoch. The purpose of this is to increase the learning rate by starting the learning rate from a large value. The learning rate was reduced throughout the training to avoid the risk of not reaching the global optimum value.

The categorical cross-entropy loss function, which is a commonly used metric in multiple classification problems, was used on the validation data set. The batch size was determined as 16. With the validation dataset, the loss, accuracy, precision and recall rates of our model were observed during training. The dataset used within the scope of this study, its features and the distribution of training, validation and test datasets are specified in Section 3.4.

3.4 Dataset

Table 2. Distribution of OCT dataset

Some examples of the dataset are shown in Figure 12.

12.png

Figure 12. Sample images from OCT dataset: a) Healthy; b) CNV; c) DME d) DRUSEN

The public OCT image dataset presented in the study of Kermany et al. [31] in 2018 was used in this study. They collected these images from five institutes from 2013 to 2017. The images in the dataset are in 2-dimensional JPEG format, and the image resolutions are different. The image data was reviewed by retinal specialists, ophthalmologists, and medical students. As shown in Table 2, the dataset contains 84,484 OCT scans. The data were split into 1,000 test images, 4,000 validation images, and 79,484 training images. Of the 79,484 training images, 25,315 were labeled Healthy, 36,205 CNV, 10,348 DME, and 7,616 Drusen. Test data consists of 250 images from each class, and validation data consists of 1,000 images from each class. Images are divided into folders in JPEG image format according to their tags.

This paper proposes new results on retinal diseases detection using deep learning methods. OCT images were used to perform retinal disease classification problem. Nine different deep learning models and our model EffXceptNet were applied to OCT dataset to classify CNV, DME, and drusen patients and healthy individuals. It was observed that better performances were obtained after hyperparameter tuning performed to the discussed deep learning models. As the performance analysis results show, the proposed hybrid deep learning model outperformed the other models. The classification success of the proposed hybrid model obtained as 99.90%. This result shows the effectiveness of the proposed model in detecting retinal diseases.

This work can be turned into software system as an automatic disease diagnosis tool that can be used by experts in medical departments. In addition, mobile applications can be developed to help general practitioners make quick decisions about retinal diseases. Thus, the proposed EffXceptNet model may shorten the clinical diagnosis time and contributes to the early referral of emergency patients to specialist physicians. Early treatment prevents the progression of retinal disorders. In addition, the costs of health services are reduced by intervening in the treatable disease earlier instead of costlier interventions in the advanced stages of the disease [71, 72]. In addition, models that detect more retinal diseases can be developed besides CNV, DME and DRUSEN diseases we detected in this study. The proposed model can be expanded for to give successful results when different medical image sets are trained with this model. That means the proposed approach can be used to detect different kind of diseases. First of all, a data set containing the diseases to be addressed must be obtained. After this data set is trained on the model we recommend and the necessary optimizations are made, the system will work to diagnose the diseases on which it is trained.