Lung Cancer Classification Using Squeeze and Excitation Convolutional Neural Networks with Grad Cam++ Class Activation Function

When it comes to the top cause of mortality in the world [1], lung cancer has surpassed all other cancers, accounting for the biggest number of deaths. According to cancer data, there are expected to be 142,670 lung cancer deaths in the United States in 2019, accounting for 23.51% [2] of all cancer patients in the country. Patients' 5-year survival rates climb from 19% to 54% when lung cancer is detected early, according to the American Lung Association [3]. As a result, early identification of lung cancer is crucial in order to maximize survival chances for patients. Typically, a nodule in the lungs is characterized by the presence of parenchymal lesions that are less than 3 cm [4] in diameter and that are spherical in shape. Figure 1 displays the categorization of lung nodules into two types: those that are malignant and those that are benign. The majority of the time, medical pictures such as magnetic resonance imaging (MRI), positron emission tomography [5] (PET), and computed tomography [6] (CT) scans may identify lung nodules (CT). When it comes to recognizing [7] and detecting nodules, low-dose CT has emerged as the most sensitive and extensively utilized imaging technology. The great spatial resolution, non-invasive nature, and cheap cost of CT screening [8] are the three most important advantages of this procedure. The appearance of nodules [9] as a regular or irregular white shadow on CT scans increases their visibility, increasing the likelihood that they will be discovered [10]. In a similar vein, the National Lung Screening Trial [11] (NLST) discovered that CT screening reduced lung cancer mortality [12] by 20%.

In order to arrive at an appropriate interpretation of the nodule, radiologists must analyse massive quantities of nodule data margin, scope, consistency, and outline, among other things from several CT scans, which is a complex and time-consuming job. Also contributing to a misunderstanding of the visual data are circumstances such as exhaustion, inattention, and a lack of professional skill, to name a few examples. Systemic computer-aided diagnostic [13] (CAD) systems, which comprise computer-aided detection (CADe) and computer-aided diagnosis (CADx) systems, have shown to be a very successful tool for the detection and diagnosis of lung cancer in a number [14] of studies. This research focuses on computer-aided diagnosis (CADx) systems, which make use of computed tomography (CT) images to identify and diagnose potentially harmful abnormalities. There are four primary processes in CADx systems, which are widely used in the pharmaceutical business and consist of data collecting, nodule segmentation, feature extraction, and nodule classification, among other things. When it comes to nodule diagnosis, one of these approaches, feature extraction, is quite useful. Overall, there are two types of nodule classification methodologies that are now accessible, and they may be alienated into 2 groups. Both malevolent and benevolent nodules may be distinguished using handmade characteristics retrieved using classic feature descriptors, while deep features learnt using different deep convolutional neural networks may be used to distinguish malignant and benign nodules as well (DCNNs).

1.jpg

Figure 1. The benign and malignant nodules in LUNA 16 benchmark dataset

The proposed technique for the classification of benevolent and malevolent lung nodes is divided into 3 major stages. These are as follows:

(1) Image data gaining from the “LUNA16” database.

(2) Node Cataloguing using Squeeze and Excitation extraction of nodules.

(3) Feeding classified image to Gard-Cam++ Activation function for the foreseeable future classification.

3.1 Proposed model

3.1.1 Data acquisition from LUNA-16 benchmark dataset

This study uses the Lung Image Database Consortium image collection (LIDC–IDRI) dataset, which has been upgraded from the freely accessible Lung Image Database Consortium dataset (LIDC–IDRI). Thin-slice CT scans are optional by the American College of Radiology for nodule identification and categorization. Because of this, scans having slice thicknesses larger than 3 mm, images with uneven slice spacing, and scans with missing slices were rejected. Another precaution, given that physicians' experience has shown that tiny nodules are usually innocuous, is to remove any nodules with a diameter of less than 3 mm simply to be careful. We discovered 888 CT images with 1004 nodules and selected all of them, yielding a total of 888 CT images with 1004 nodules. Four radiologists with extensive experience provided comments for each CT image. The annotations in the dataset were acquired as part of a two-phase image annotation methodology that was acknowledged by at least three out of four radiologists in the study. most specific characteristics, such as node positions, diameters, and malevolence stages (from 1 to 5). "A cancer risk of Level 1 indicates that there is no chance of developing cancer; a cancer risk of Level 2 indicates that cancer development is only possible; a cancer risk of Level 3 indicates that disease development is slightly likely; a cancer risk of Level 4 indicates that disease development is more likely; and a cancer risk of Level 5 indicates that disease development is very likely."

3.png

Figure 3. Background of the anticipated node diagnosis method

3.1.2 Data extraction

To ensure that the learning model is correctly trained, 3D patches containing lung nodules are clipped from raw CT images and inserted in the network to serve as training examples. In the centre of each nodule is a cantered patch of fungus. Expert radiologists estimate that the two greatest diameters of the nodules that were selected are 33.26 mm and 31.62 mm, with the other node sizes ranging from 3.1 mm to 35 mm. Nodule sizes were chosen at random from the LUNA16 database, and the distribution of nodule sizes is shown in Table 1. Nodules that are not cancerous are often smaller in size than nodules that are cancerous; however, this is not always the case. Due to the large number of nodules that were chosen, the scope of the contribution patch has been expanded to 32*32*32 in order to capture the majority of them chosen. Figure 3 shows the brief view of our proposed model. Diagram Showing Data acquisition, Node Extraction and Nodule Classification to check weather it is Benign or Malignant.

Table 1. The spreading of node dimensions in the LUNA16 database

3.1.3 Basic structure of squeeze-and-excitation networks

SENETS are a type of CNN that, despite having limited computational power, increases channel interdependence. SENETS are a type of CNN that is subdivided. Through the use of channel adjustments for each conversion block, we were able to fine-tune the weighting on each feature map. Using these filters, CNNs are able to extract hierarchical information from images. Higher layers are capable of detecting faces, text, and other complex geometric shapes, whereas lower layers are only capable of detecting borders or extremely high frequencies. They gather all of the information necessary to complete a task in a short period of time. This is accomplished through the combination of space and image data. Following that, the filters will combine data from all of the output channels that are currently available. When the network generates output charts, it assigns the same weight to each of the channels. SENets intends to alter the channel weighting by incorporating a content-aware mechanism into its network. In its most basic form, this entails assigning a single parameter and linear scalar to each of the channels.

Figure 4 shows the Squeeze and Excitation Network and working of the proposed Model for classification of the Lung Cancer Nodules.

In convolutional neural networks (CNNs), SENETS (Squeeze-and-Excitation Networks) are a novel kind of building component that enhances channel dependence while requiring low computational effort. As a reminder, the basic notion is as follows: A convolutional block should have parameters assigned to each of the channels, which will allow the network to adjust the weighting of each feature map as necessary, as shown in the example. When CNNs analyse images, convolutional filters are used to collect hierarchical information contained within the images. Higher degrees of intelligence may be able to recognise faces, language, and other complex geometrical patterns that are undetectable to lesser levels of intelligence. Figure 5 shows the detailed view of the proposed Model.

They gather whatever information is necessary in order to complete a task properly. By merging the spatial and channel data contained inside a single image, this may be done. The different filters will search for spatial components in each individual input channel before aggregating the data over all available output channels to get a final conclusion. When the network is constructing the output feature maps, it provides equal weight to each of the channels that it has. SENETS objective of revolutionising the Internet involves including a content-aware technique for adaptively weighing each channel as part of its overall objective of revolutionising the Internet. In its most basic form, this may be accomplished by assigning a single parameter to each channel and processing it as if it were a linear scalar value. For each channel, they begin by condensing its feature maps into a single numerical value, which they then use to get a global understanding of that channel. This results in an n-dimensional vector, where n is the number of convolutional channels in the underlying technique used to generate the vector. Data is then fed into a two-layer neural network, which generates a result with a size equal to or greater than that of the input vector. Now that the n values have been determined, they may be used as weights on the original feature maps, allowing for accurate scaling of each channel.

4-1.png

Figure 4. The basic structure of the squeeze and excitation network for the lung cancer classification

5.png

Figure 5. The structure of proposed network SENETS

• The function is given an input convolutional block as well as the current number of channels available.
• We reduce each channel to a single numeric quantity using average pooling.
• The required nonlinearity is achieved by adding a fully connected layer followed by a Mish function. 3. Furthermore, the production channel complication is reduced by a certain amount.
• A smooth gating function for each of the eight channels is provided by a additional entirely linked layer surveyed by Sigmoid activation.
• We weight respectively convolutional block feature map based on the results of our side network.

3.1.4 SENETS with Grad Cam++ Class activation

According to this research, we can use this method to offer visual explanations by highlighting discriminative areas in the model, which is a very strong strategy for finding out how to make the model interpretable for those who are not familiar with the domain. Essentially, we want to develop a 3D CNN with the highest level of accuracy possible while simultaneously providing visual insights into the model's decision-making processes. This study proposes grade cam++ techniques for visual nations based on a single module categorization, which is shown by a single module classification in this study. Grad-CAM++ is supported by a large number of different CNN models. It generates a visual image that distinguishes between different classes of people. It also informs you of the model's failure modes, should they occur. To determine the importance of each neuron for a particular decision, Grad-CAM++ examines the information on the gradient that is passed into the final convolutional layer of the CNN, which is called the final convolutional layer.

$Y_{G r a d-C a m}^{a}=\sum_{p}^{1} w_{p}^{q} \frac{1}{z} \sum_{i} 1 \sum_{j} A_{i j}^{p}$       (1)

Eq. (1) is used to calculate the average pooled global Output of the Squeeze and Excitation Network.

Let us define J^P to be average pooled global output

$J^{p}=\frac{1}{z} \sum_{i} * \sum_{j} A_{i j}^{p}$       (2)

Grad-Cam++ Computes the end scores by gradient via back propagation of the network J^P is the object of the prediction and their confidence of the image, nodule classification with discriminative regions, object annotations.

$Y_{G r a d-C a m}^{a}=\sum_{p}^{1} * w_{p}^{q} * J^{p}$       (3)

where, $w_{p}^{q}$  is the weight connecting the Kth Feature map with forward activation maps with non-linearity to the squeeze and Excitation Network.

Taking the gradient class score (J^P) w.r.t to feature map we get.

$\frac{\delta y_{1}^{c}}{\delta J_{1}^{p}}=\frac{\frac{\delta y^{c}}{\delta A_{i j}^{p}}}{\frac{\delta J^{p}}{\delta A_{i j}^{p}}}$       (4)

Taking Partial Derivation (4) w.r.t. $A_{i j}^{p}$ ; We can see that $\frac{\delta J^{p}}{\delta A_{i j}^{p}}=\frac{1}{z}$ . Substituting this on (4) we get Grad-CAM++, which uses a weighted combination of the positive partial derivatives of the last convolutional layer feature maps with respect to a specific class score as weights to generate a visual explanation for the class label under consideration.

$\frac{\delta y_{1}^{c}}{\delta J_{1}^{p}}=\frac{\delta y^{c}}{\delta A_{i j}^{p}} * z$       (5)

From (3) We get

$w_{p}^{q}=z * \frac{\delta J^{p}}{\delta A_{i j}^{p}}$       (6)

Summing both sides of (6) we get to know the $Z * \frac{\delta J^{p}}{\delta A_{i j}^{p}}$  visualize Convolutional Networks and understand what is learnt by each neuron.

$\sum_{i} \sum_{j} W_{p}^{q}=\sum_{i} \sum_{j} z * \frac{\delta y^{c}}{\delta A_{i j}^{p}}$       (7)

Therefore $\frac{\delta y^{c}}{\delta A_{i j}^{p}}$  is used for the pixel wise Value and the true value of the lung nodule.

$w_{p}^{q}=\sum_{i} \sum_{j} \frac{\delta y^{c}}{\delta A_{i j}^{p}}$       (8)

Because of this, Grad-Cam++ may be considered to be a rigorous generalisation of the class activation. It is possible to offer a visual explanation for domain experts by creating a 3D CNN-based model, which makes the procedure both lighter and more sophisticated due to this generalisation. As a consequence, computer tomography scans will be able to pinpoint the exact site of the tumour. To construct a Mish, we do a weighted combination of forward activation maps followed by a ReLu.

$L N_{G r a d-C a m}^{a}=\operatorname{Re} L u\left(\sum_{k} \alpha_{k}^{c} A^{k}\right)$       (9)

From the Eq. (9), we pointwise multiply the up-sampled (to image resolution) saliency map Lc with the pixel-space visualisation generated by Guided Backpropagation in order to generate class-discriminative saliency maps with the richness of pixels-space gradient visualisation methods. Grad CAM++ makes use of a technique that is similar. Guided Representations using Grad-Cam++ are the end result of this procedure.

SEGNET is being investigated and developed with the goal of improving the accuracy and confidence in the identification of lung nodules and CT scan data compared to current models, with the ultimate goal of aiding in the early diagnosis of lung cancer in order to enhance survival and prognosis.

3.1.5 Class discrimination and trust

These methods are each devolutionised, with redirected background and Grad-cam++ versions available (Deconvolution Grad-CAM and Guided Grad-CAM). It is possible to visualise 160 different images of the lung nodule in order to answer the question "What image represents the two categories of objects?" During the testing process, a total of 90 pairs of images (360 visualisations) were used. The following 9 people have given their opinions. The category visualised is correctly identified by humans when they view Grad-cam++ (63.43% when viewing Guided Backpropagation, compared to 44.44% when viewing Grad-CAM++; thus, Grad-cam++ improves human performance by 16.79%). Grad-CAM++, on the other hand, would appear to be more discriminatory in its treatment of Deconvolution (from 54.36% to 62.32%). Grad-cam++ outperforms all of the other methods. It was taken into consideration only when the two models predicted the same outcome as the fundamental truth. Using 54 lung nodules, the model's reliability was assessed using three different levels of reliability: outstandingly more/unreliable (+/-2), slightly more/unreliable (+/-1), and equally reliable (+/-1). (0). Despite the fact that the predictions are identical, noduled subjects can identify the more accurate classifier by examining different explications.

3.1.6 Nodule classification network parameters

Figure 6 displays a SENETS network with Grad Cam++ class activation for the classification of lung nodules. A total of 32 x 32 x 32 image patches is supplied into the network as input. For the first time, a convolutional level is used in the generation of the feature map (3 x 3). Table 2 shows the Parameter of the Proposed Classification Network

In order to get the necessary results, SENET blocks are interposed with the average-pooling layers, which are interleaved with four average-pooling layers, which are interleaved with four average-pooling layers. Three Squeeze and Excitation modules are included inside each of the four Squeeze and Excitation blocks (the first, second, and third), with the first block containing four modules, the second including six modules, and the third including three modules. Using a average-pooling layer, each block is segregated from the rest of the blocks. The last layer of the network is composed of three layers, which are arranged in the following sequence: an average pooling layer, an FC layer, and a SoftMax layer (in that order). It is shown in this study how to solve a binary classification issue using the cross-entropy loss function. Following the SoftMax, the projected malignant probability (PMP) of each lung nodule may be estimated individually.

6.jpg

Figure 6. The SENETs with Grad Cam++ Class activation

7.png

Figure 7. The proposed architecture

Table 2. The parameters of the proposed network

In order to evaluate the methodology, the researchers employed the LUNA16 nodule database, which has 1004 nodules. The image data was separated into 10 subgroups at random, with each subgroup being cross-validated 10 times. As a result, all of the image data was divided into 10 subsets that were distributed equally and randomly. It was necessary to analyse nine sets of training data in order to determine which ones should be used, however only one set of testing data was assessed throughout the process. This method of classifying nodule-level trials into binary categories included averaging the levels of each nodule and labelling them as questionable or certain depending on whether the final average level was more than or equal to 3. The presence of a malignant nodule is determined by the average final level of the nodule being more than 3. However, there is no scientific evidence to suggest that they are harmful to humans. The LIDC–IDRI database was used to collect all of the annotations for the item in question. The chance of malignancy was projected to aid radiologists in their diagnosis, and in addition, Windows 10 and Ryazan 9 series CPUs, as well as 12 GB graphics cards, were used in each of the tests.

5.1 Implementation details

9.png

Figure 9. Classification accuracies of diverse frameworks with comparison with different classifiers

Figure 9 shows the performance evaluation of the proposed model with other classifiers.

Several CT scanners were utilised in the same experiment to analyse images from the LUNA16 database, which was conducted on the same day. Before the training method, all CT scans were resampled to the same exact resolution of 1 mm, which is one millimetre by one millimetre: one millimetre by one millimetre. To scale the pixel values of the clipped region, they were first normalised to the range [0, 1] and then truncated to the range [1200, 600]. For the purpose of reducing overfitting, data augmentation was performed to the training dataset, which was checked for accuracy by randomly flipping and rotating the patches on the training dataset. The stochastic gradient descent algorithm was used to optimise the learning process (SGD). The process had an impetus of 0.9, and its weight decay was more than 1,000 times more than that of the process. Furthermore, since there was a restriction on the amount of GPU RAM that could be used, the batch size was restricted to a maximum of 24 persons per batch. A total of 150 training sessions were conducted using the learning model. Initially, the initial learning rate was 0.001, and after 60 epochs, it was 0.0001, and after 120 epochs, it was 0.0001. Using the AUC score and accuracy calculations, as well as the average performance on each of the 10 test folds, the suggested methodology was assessed. As for the accuracy of lung nodule diagnosis, it is defined as the fraction of correctly identified nodules, and it is provided in the following as:

$\text { Accuracy }=\frac{T P+T N}{T P+F N+F P+F N}$       (11)

$\text { Sensitivity }=T P /(T P+F N)$       (12)

$\text { Specificity }=T N /(T N+F P)$       (13)

$\text { Precision }=T P /(T P+F P)$       (14)

$\begin{aligned}

&F P R=F P /(F P+T N) \\

&T P R=T P /(T P+F N)

\end{aligned}$       (15)

where, the TP is true positive, TN is true negative, FP is false positive, and FN is false negative.

From the Table 3, it was very clear that our classifier SENETS-Grad-Cam++ has outperformed remaining all the classifiers in terms of the performance metrics like AUC and accuracy on LUNA 16 benchmark dataset.

Figure 11 and 12 Shows the Model Classification after the Testing. The red heat map shows the benign lung nodule classification. Thus as a results it would be easy for the radiologists to classifier the tumor.

From the Table 4 it was clearly evident that our method was performed very well with few samples size from LUNA 16 dataset. The results clearly shows that out proposed model and method are good at classifying odd benign and Malignant tumours. As, a result the model has achieved a trust gain when compared with various others methods. From the Figure 10 we can clearly state that with ROC curve performance our model has out performed when compared with other classifiers.

10.png

Figure 10. ROC curve showing the proposed classifier

11.jpg

Figure 11. Visualization of benign and malignant nodule classifications. Malignant prognosis, four radiologists, and nodule diameter (mm)

12.jpg

Figure 12. Grad Cam++ visualization of the malignant nodules after classification

Table 3. Evaluation of proposed with other procedures based on handcrafted structures

Table 4. Assessment of the proposed methods with other approaches based on deep learning topographies