Optimizer Comparison for a GoogleNet-Based Tuberculosis Classification Model

2.1 System architecture

The initial stage in building the architecture in Figure 1 is to take photographs of TB with a microscope. This method makes sure that the data is good enough to be used for further analysis. The following step after taking the picture is preprocessing, which uses a median filter. To get rid of noise in photographs while maintaining crucial information, which is very critical for correct analysis, you apply a median filter.

tu_pian_1.png

Figure 1. System diagram

After that, the processed TB dataset will be divided into two parts: 80:20. In this part, 80% will be training data to teach the model, and 20% will be test data to see how well the model works. It is very important to split this dataset so that the model doesn't overfit, which is when it fits too closely to the training data and doesn't generalize well to new data.

Then, the next step is to use a CNN with the GoogleNet architecture to classify the data. GoogleNet was chosen since it has been shown to be able to handle complex image data and complex hierarchical features. GoogleNet uses a structure called an "inception module" that lets the network efficiently extract features from different scales. The inception module lets the network process images with different levels of resolution at the same time by using many parallel paths in one layer. This makes the visual representations richer.

After the classification process is complete, the results will be evaluated thoroughly to measure the accuracy, precision, recall and F1-score value of the model developed. This evaluation will also include the time it takes to classify the entire dataset. This analysis is important for figuring out how well Google's CNN model can classify TB image data while also considering how long it takes to do it.

2.2 Sputum images dataset

The dataset employed in this study comprises sputum images obtained using a microscope. A total of 1,266 images were successfully collected. The dataset is divided into two classes: 633 images of sputum from TB patients and 633 images from non-TB individuals. Below are examples of TB-positive and TB-negative sputum images as shown in Figure 2 and Figure 3.

tu_pian_2.png

Figure 2. Images of tuberculosis individuals

tu_pian_3.png

Figure 3. Images of non- tuberculosis individuals

2.3 Median filter on sputum images

The median filter was originally introduced by Tukey [7] and is a nonlinear filtering technique that operates by sliding a narrow window across the image, often with an odd number of pixels [8]. Compared to linear smoothing filters of the same size, it is particularly successful in removing impulsive noise, sometimes known as salt-and-pepper noise [9]. In practice, the filter sorts the pixel values in the window and replaces the center pixel with its median. This method reduces harsh changes, allowing the pixel values to merge more organically with their neighbors. Additionally, the Median Filter can also change the values of isolated pixel groups, which have brightness or darkness levels different from their neighbors and have an area less than n²/2, by using the median value from an nxn matrix. Consequently, the noise removed by the Median Filter will have values similar to the median intensity of its neighboring pixels [8].

tu_pian_4.png

Figure 4. Block diagram of the median filter workflow

Figure 4 shows that in the context of the sequence P(1) < P(2) < P(3) < P(n), the statement refers to the process of sorting data elements from the smallest rank (P(1)) to the largest rank (P(n)). In other words, each data element is sorted based on its relative value in the dataset, from smallest to largest as shown in Figure 5. For example, if we have a dataset containing values 3, 7, 1, and 5, after the sorting process, the sequence of values will be 1, 3, 5, and 7. Meanwhile, the value of m corresponds to the formula:

$m=\frac{n+1}{2}$          (1)

where, n = number of data, m = new median value.

tu_pian_5.png

Figure 5. Example of median filter application

2.4 Architecture GoogleNet

The main development offered by GoogleNet is its application to a structure known as inception model [10]. Overall, inception adopts the concept of a network within a network, consisting of sub-networks to optimize performance. In the context of image processing, an efficient local structure is rarely applied repeatedly from start to finish to obtain better feature representations. In its application, three types of inception structures tailored to different needs are introduced: generally, 1 × 1 convolutions are used to reduce dimensions before applying more complex 3 × 3 and 5 × 5 convolutions, thereby improving computational efficiency and enhancing feature representation quality. Furthermore, the use of inception modules enables the network to learn hierarchical representations from simple to complex features, with each sub-network focusing on processing different features in the image. This allows the network to be more adaptive and capable of capturing various levels of detail in image data, improving object recognition and classification performance. This network has quite impressive capabilities in classifying patterns from around 1000 images. Additionally, compared to AlexNet, GoogLeNet uses significantly fewer parameters, about 12 times fewer [11]. Like most neural networks used in computer vision contexts, this model takes images as input and produces labels for the classes it learns, along with confidence levels as output [12]. The GoogLeNet architecture consists of a total of 22 layers, which include 9 inception modules. The modified inception modules, as seen in Figure 6, utilize adaptable filters with sizes ranging from (1 × 1) to (5 × 5) to perform convolution in parallel. This approach assists in capturing various levels of detail from the existing features [13].

tu_pian_6.png

Figure 6. GoogleNet architecture

2.5 Optimization algorithm

The optimization process in machine learning plays a crucial role as an essential tool to adjust the values of the objective function based on the available data. Through consistent iterative steps, the algorithm continuously updates the model parameters to minimize prediction errors. Thus, the model can continuously learn from the available data and improve its accuracy. This study used four different optimization methods to boost the model's efficiency, which shed light on the ways in which each technique aids in the development of better machine learning algorithms.

RMSprop is a modification of the AdaGrad method that seeks to enhance performance in non-convex scenarios by altering the accumulation of gradients by the application of an exponential moving average. When utilizing a conventional learning rate of 0.001 in Stochastic Gradient Descent (SGD), RMSprop helps fix the problem of the learning rate dropping too quickly. To understand how RMSprop works better, look at how the parameters vary in Eqs. (2)-(4). Gives a general idea of how the RMSprop algorithm updates model parameters depending on the gradients that have been collected [14].

$r=\rho r+(1-\rho) g \odot g$             (2)

$\Delta \theta=\frac{\alpha}{\delta+\sqrt{r}} \odot g$             (3)

$\theta=\theta+\Delta \theta$             (4)

In this instance, r denotes the accumulation of the squared gradient, which is beneficial for modulating parameter updates by considering gradient fluctuations. The parameter p functions as the decay rate, dictating the rate at which the accumulation of the square gradient diminishes with time. The computed parameter update is referred to as ∆θ, representing the modification required for the model parameters. The learning rate, denoted by α, is the magnitude of adjustment applied at each iteration during parameter updates. The constant ẟ, valued at 107, is employed to avert division by zero in the computation. Ultimately, θ represents the first model parameter subject to modification. All these parts collaborate to ensure the model learns from the input effectively and efficiently.

A prominent variant of the gradient descent optimization technique is SGD. In contrast to batch gradient descent, which modifies parameters solely after processing the complete dataset, SGD updates parameters following each unique training example. This regular updating enables the algorithm to converge more rapidly, rendering it particularly advantageous for extensive datasets. Due to its efficiency and responsiveness, SGD is frequently utilized in situations requiring swift parameter modifications, typically with a learning rate of 0.01. The SGD parameter update rule is presented in Eq. (5), which demonstrates the iterative optimization of model parameters utilizing the gradient of an individual data point at each iteration.

$\theta=\theta-\eta * \nabla_\theta \mathrm{J}\left(\theta ; x^i ; y^i\right)$            (5)

Here, θ indicates the parameter that is modified with each iteration of model training, whereas η denotes the learning rate, which determines the magnitude of the adjustment. The pair xi represents the input data and the output data utilized in that phase. In each iteration, the model refines θ utilizing the gradient derived from the data, so enhancing the parameters incrementally with each cycle. In this manner, SGD facilitates the model's continuous enhancement by learning from individual examples, enabling rapid adaptation and effective responsiveness to variations in the training data [14].

Adam is a widely utilized optimization algorithm that integrates concepts from two proven techniques: RMSProp and Momentum. RMSProp incorporates the capability to automatically modify the learning rate for each parameter, hence addressing the prevalent challenge of selecting an appropriate learning rate. Simultaneously, it employs the concept of momentum to ensure updates go in a consistent manner, rather than changing too abruptly. By integrating these two methodologies, Adam renders the optimization process both stable and efficient. This has rendered it one of the most prevalent algorithms in machine learning, facilitating robust performance across various tasks.

$m_i=\beta_1 m_i+\left(1-\beta_1\right) \frac{\partial L}{\partial \theta_i}$         (6)

$v_i=\beta_1 v_i+\left(1-\beta_2\right)\left(\frac{\partial L}{\partial \theta_i}\right)^2$            (7)

In this context, parameter (6) refers to momentum, while parameter (7) represents the exponential moving variance used in the optimization algorithm. Momentum (6) helps speed up convergence by keeping the parameter updates moving in a consistent direction. On the other hand, the exponential moving variance (7) adapts the learning rate for each parameter, making the optimization process more stable and efficient. When the initialization values for time steps and decay rates are too small, parameters (6) and (7) tend to approach a value close to 1, which can introduce bias in the estimation. To mitigate this issue, bias correction and moment estimation are conducted through the division of the parameters in Eqs. (6) and (7) by the difference between 1 and the decay factor. This adjustment stabilizes the moment calculations and produces more accurate estimates.

Implementing this adjustment helps minimize the initial estimation bias, leading to more dependable parameter values throughout the optimization process. This step is essential to ensure consistent and dependable results, especially when dealing with sensitive initial conditions.

$\widehat{m}=\frac{m_i}{1-\beta_1}$              (8)

$\hat{v}=\frac{v_i}{1-\beta_2}$            (9)

Adam, an optimization algorithm proposed by its developers, suggests setting β₁ to 0.9, β₂ to 0.999, and ε to 10⁻⁸. These values were determined through extensive experiments to provide strong performance across a wide range of tasks. In this setup, beta-1 and beta-2 control the exponential moving averages of the gradient and the squared gradient, while epsilon is included to prevent division by zero. After the optimal values for parameters (8) and (9) have been obtained, which represent these moving averages, the Adam update formula can then be applied.

$\theta_{t+1}=\theta_t-\frac{\partial}{\sqrt{v_t+\varepsilon}} \cdot m_t$           (10)

According to its formulation (10), Adam combines the core ideas of RMSProp with the momentum method in gradient estimation to improve both speed and stability during model training. By using the exponential moving average of the gradient together with the exponential moving variance, Adam can adjust the learning rate for each parameter individually. This adaptability gives Adam an advantage over earlier optimization algorithms [15-17].

SGDM is an adaptation of the gradient descent optimization technique that modifies model parameters during the processing of training data. SGDM is distinguished by the use of a momentum factor, which accelerates convergence and mitigates oscillations, especially in the context of extensive datasets. SGDM changes parameters by integrating momentum from prior updates, enabling the model to navigate more smoothly through both flat and steep gradient areas, rather than solely relying on the complete dataset for updates. The standard learning rate for SGD is typically established at 0.01, but this figure may be modified based on the intricacy of the issue.

$\mathrm{Gt}=\nabla_\theta \mathrm{J}\left(\theta ; x^i ; y^i\right)$           (11)

In SGDM, the parameter θ is revised at each step. The learning rate η dictates the magnitude of each update step, whereas xⁱ and yⁱ denote the input-output data pairs utilized in training. With each iteration, θ is modified according to the processed data, enabling the model to progressively enhance its performance by learning from each instance [18].

2.6 Confusion matrix

Figure 7 illustrates how the four mentioned values are arranged in the confusion matrix. This version also lists the total number of observations in the top left corner, as well as the marginal counts at the top and left side of the matrix. The colors used for each section of the matrix represent the combination of categories that form the values within it [19]. For example, the color green represents true positives, which consist of false positives and false negatives. Each model has four values that describe its own performance. Therefore, the importance of each of these values must be considered when comparing the overall performance of the models [20]. Classification metrics are calculated from the numbers in the confusion matrix, so to understand these metrics, one must understand the meaning of each value within it [21].

tu_pian_7.png

Figure 7. Confusion matrix

True Positive: This means the model correctly detected the disease.

True Negative: This means the model correctly identified that there is no disease.

False Positive: This means the model incorrectly identified someone as having the disease when they actually do not.

False Negative: This means the model failed to detect someone who actually has the disease.

With a confusion matrix, various model performance evaluation metrics can be calculated, such as accuracy, precision, sensitivity (recall), specificity, and F1-score [22]. A confusion matrix is a very useful tool for understanding the performance of a classification model and identifying areas where the model needs improvement. A brief explanation of some commonly used evaluation metrics alongside the confusion matrix [23]:

Accuracy

The percentage of total correct predictions made by the model out of all the test data.

$Accuracy =\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}}$         (12)

Precision

The percentage of true positive predictions out of all positive predictions made by the model.

$Precision =\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}}$         (13)

Recall

The percentage of positive data successfully identified by the model out of all the actual positive data.

$Recall =\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}}$         (14)

F1-Score

The harmonic mean between precision and sensitivity. It is used to compromise between both in one metric.

$F1-Score =2 \mathrm{TP} /(2 \mathrm{TP}+\mathrm{FN}+\mathrm{FP})$         (15)

The conclusion from the experimental results indicates that the GoogleNet model excels in TB classification. For this classification task, the Adam optimizer was utilized with a batch size 16, a learning rate of 0.0001, and a total of 20 training epochs. With a loss rate of 5.80% and an accuracy rate of 98.52%, the model demonstrates excellent capability in accurately classifying TB images. Additionally, the model has high precision, recall, and F1-Score, each reaching 98.52%. The performance graph of GoogleNet loss shows a decrease in loss values on the training data from epoch to epoch, indicating the model's progress in understanding the data patterns. Despite some variation in the validation loss, the model's performance remains high with good accuracy. Overall, the graph illustrates the improvement in GoogleNet's accuracy in classifying TB data as the iterations progress.

Future research will focus on implementing layer selection techniques in the CNN architecture to reduce training time and computational costs, while simultaneously improving the overall accuracy and robustness of the model.